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CHAPTER  I 


INTRODUCTION 

Improving  the  safety  of  complex  human-machine  systems  is  a 
continuing  challenge.  Available  information  concerning  system 
failures,  which  are  usually  called  accidents,  incidents,  or  mishaps, 
regularly  points  to  human  operators  as  the  '•brittle  elements."  (5:1) 
The  need  for  greater  understanding  of  operator  behavior  is  recognized 
in  a  variety  of  technologically  sophisticated  systems,  for  example, 
industrial  processes,  health  care,  public  utilities,  and  national 
defense;  but  nowhere  is  it  more  obvious  than  in  the  aftermath  of  a 
commercial  airline  accident. 

Following  the  November  1979  crash  of  an  Air  New  Zealand  DC-10 
which  killed  257  people  in  Antarctica,  United  Press  International 
noted  that  the  ten  worst  disasters  in  aviation  history  have  all 
occurred  since  October  1972.  (61:4)  Although  the  degree  of  operator 
culpability  varies,  the  fact  that  the  five  deadliest  accidents  have 
occurred  since  March  1974  is  particularly  significant.  At  least  one 
commercial  airliner  was  completely  destroyed,  and  over  200  people 
were  killed  in  each  of  the  five  crashes.  Wide-body  jetliners,  either 
Boeing  747  or  McDonnell  Douglas  DC-10  aircraft,  were  involved  in  each 
instance.  The  sheer  size  of  such  vehicles  portends  grave  conse¬ 
quences  in  case  of  system  failure,  be  it  human,  mechanical,  or  a 
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combination  of  the  two.  Despite  steadily  decreasing  rates  for  total 
air  carrier  accidents  (cf.  Table  1),  when  a  system  failure  does  occur, 
the  potential  for  tragedy  is  great. 


TABLE  1 

U.S.  AIR  CARRIER  ACCIDENT  RATES 
ALL  OPERATIONS  1967-1978  (1,33) 


Year 

Accidents 

Accident 

100,000 

Hours 

Rate  Per 
Aircraft 
Flown 

Accident  Rate  Per 
Million  Miles  Flown 

Total 

Fatal 

Total 

Fatal 

Total 

1967 

70 

12 

1.193 

.204 

.032 

1968 

71 

15 

1.109 

.203 

.028 

1969 

63 

10 

.935 

.134 

.023 

1970 

55 

8 

.850 

.124 

.020 

1971 

48 

8 

.752 

.094 

.018 

1972 

50 

8 

.793 

.127 

.019 

1973 

43 

9 

.661 

.138 

.016 

1974 

47 

9 

.769 

.134 

.019 

1975 

45 

3 

.745 

.050 

.018 

1976 

28 

4 

.450 

.064 

.011 

1977 

26 

5 

.404 

.078 

.010 

1978 

26 

6 

.389 

.088 

.009 

Rationale  for  Research 

For  many  years  safety  conscious  individuals  from  nearly  all 
segments  of  the  commercial  aviation  community — manufacturers,  govern¬ 
ment  regulators,  airline  executives,  accident  investigators,  air 
traffic  controllers,  pilots — have  recommended  patchwork  modifications 
within  the  system.  They  have  concentrated  on  the  accident  potential 
of  traffic  flows,  information  displays,  mechanical  reliability,  warn¬ 
ing  devices,  and  similar  components  of  the  total  system  which  could 
be  logically  segregated.  As  a  consequence,  regulatory  and  engineering 
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changes  haw  been  Imposed  on  aircrews  almost  continuously.  Inertial 
navigation  systems,  computerized  flight  plans,  low  visibility  approach 
criteria,  and  Standard  Terminal  Arrival  Routes  (STARs)  are  just  a  few 
of  the  modifications  affecting  routine  operations  during  the  past 
decade.  Procedures  have  proliferated  with  virtually  every  change  and 
every  accident. 

If  accident  statistics  are  meaningful  in  this  context,  then 
definite  progress  has  been  made.  From  1967  through  1978  the  number 
of  accidents  involving  U.S.  air  carriers  decreased  markedly.  As  pre¬ 
viously  implied,  however,  the  rate  of  fatal  accidents  actually  shows 
an  increase  for  each  year  since  1975.  The  chronological  data  are  con¬ 
tained  in  Table  1. 

During  the  past  five  years,  growing  concern  has  been  focused  on 
the  need  to  improve  human  performance  aspects  of  the  man-machine  inter¬ 
face.  Expanded  knowledge  of  operational  behavior  is  a  logical 
precursor  to  improvements.  In  November  1975  at  the  Air  Transport 
Association's  conference  on  "Safety  in  Flight  Operations,"  the  study 
of  basic  human  behavior  and  human  limitations  was  accorded  top 
priority  for  enhancing  safety.  (4:2)  In  1978  the  Air  Line  Pilots  Asso¬ 
ciation  (ALPA)  issued  "A  Statesmanlike  Challenge"  to  the  scientific 
community  to  advance  the  state  of  knowledge  regarding  operator  work¬ 
load  in  airline  cockpits.  (18)  Throughout  the  period,  the  National 
Aeronautics  and  Space  Administration  (NASA)  has  been  supporting  a 
Flight  Management  Research  Program  (24:411)  aimed  at  developing  greater 
understanding  of  crew  roles,  interpersonal  relations,  and  human-machine 
interactions  i  .  commercial  air  transportation.  The  research  reported 


in  this  dissertation  is  one  outgrowth  of  an  experiment  conducted  in 
1976  by  Dr.  H.  P.  Ruff ell  Smith  under  the  auspices  of  the  NASA-Ames 
Research  Center.  (53) 
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Research  Objectives 

That  1976  study  signaled  the  beginning  of  a  new  era  in  aircrew 
research,  full  mission  simulation.  The  present  study  critically 
examines  the  earlier  effort  and  utilizes  some  of  the  original  data  to 
depict  routine  cockpit  behavior  in  substantial  detail.  The  emphasis 
here  is  on  the  customary  tasks  of  flight  operations,  not  rare  or 
unusual  tasks. 

The  following  chapters  address  seven  central  objectives.  In  brief, 
these  are:  (1)  definition  of  a  finite  set  of  routine  crew  procedures, 

(2)  classification  of  the  routine  procedures  into  meaningful  subsets, 

(3)  quantification  of  operator  compliance  with  one  selected  subset  of 
routine  procedures,  (4)  subjective  evaluation  of  the  managerial  talent 
of  crew  commanders,  (5)  attribution  and  enumeration  of  previously 
identified  crew  errors  to  specific  operators,  (6)  comparison  of  the 
measures  of  procedural  compliance,  management  skill,  and  operator  error, 
and  (7)  evaluation  of  full  mission  simulation  as  a  means  of  studying 
the  operational  behavior  of  aircrews. 

The  routine  or  normal  operating  procedures  are  culled  from  the 
Aircraft  Operating  Manual,  the  Company  Operations  Manual,  the  Federal 
Aviation  Regulations,  crew  handbooks,  and  assorted  navigational 
documents.  They  are  set  against  a  backdrop  of  simulated  airline  flight 
by  fully  qualified  crews  on  a  Boeing  747.  Taxonomies  of  normal 
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procedures  are  based  upon  various  characteristics  of  published  form, 
sequence,  and  content.  A  subset  called  Crew  Coordination  Procedures 
is  chosen  as  a  standard  for  documenting  typical  operator  compliance. 

The  perpetrators,  circumstances,  and  frequencies  of  different  types 
of  noncompliance  portray  the  status  of  standardization  within  a  sample 
of  one  airline's  active  crew  force.  Routine  operational  behavior  has 
not  previously  been  described  at  this  level  of  detail. 

Subjective  evaluations  of  managerial  skill  are  based  upon  three 
established  precepts  of  sound  management:  continuity,  cooperation,  and 
discipline.  Comparisons  among  the  various  quantitative  and  qualita¬ 
tive  assessments  of  operator  behavior  illustrate  the  value  of  multi¬ 
dimensional  measures  of  human  performance.  Finally,  the  usefulness  of 
full  mission  simulation  as  a  data  collection  methodology  is  considered, 
and  its  first  experimental  application  is  reviewed. 

Fundamental  Definitions 

While  some  isolated  aircrew  tasks  and  standard  operating  pro¬ 
cedures  (SOPs)  have  received  attention  in  other  studies  (59,60,69), 
routine  procedures  have  been  slighted.  Generic  classes  of  ordinary 
procedural  activity  have  been  almost  wholly  ignored  (67) .  For  some 
readers  the  words  "task"  and  "procedure"  may  have  colloquial  overtones 
which  differ  from  their  intended  sense  in  this  manuscript.  The 
definitions  and  explanations  offered  below  are  meant  to  alleviate 
possible  confusion.  They  have  been  developed  from  concepts  embodied 
in  a  paper  by  Funk  and  Miller  (16)  and  from  discussions  with  the 


authors. 


A  task  is  simply  a  collection  of  goal-oriented  behaviors.  At  the 
most  elementary  level,  any  required  behavior  could  be  thought  of  as  a 


task  whose  goal  is  subordinate  to  or  subsumed  by  another  goal  at  a 
higher  level  of  abstraction.  Throughout  this  text  the  aggregation  level 
of  goals/tasks  will  be  completely  determined  by  the  contents  of  the 
procedures  presented  for  their  accomplishment.  As  a  result,  some 
tasks  may  have  relatively  few  sub-tasks  (e.g.,  altitude  callout  during 
a  climb)  while  others  may  contain  so  many  sub-tasks  that  procedures 
within  procedures  are  provided  (e.g..  Predeparture  Planning  as 
described  in  Chapter  III) . 

What,  then,  is  a  procedure?  It  is  a  symbolic  and  mnemonic 
representation  of  a  set  of  sensory,  cognitive,  and/or  motor  activities 
which,  when  recalled  and  executed  within  determinable  tolerances,  com¬ 
plete  a  task  as  designed.  A  given  operating  procedure  need  not  be  the 
"best"  one  or  the  only  one  appropriate  to  a  particular  task.  But,  at 
least  in  airline  transportation  systems  there  exist  explicit  SOPs  pre¬ 
scribed  for  a  whole  class  of  operators  across  a  range  of  environmental 
conditions.  In  this  context  the  phrase  "standard  operating  procedure" 
is  surely  apropos. 

Organization  of  the  Dissertation 

Two  major  areas  of  research  relating  cockpit  SOPs  and  behavior 
have  been  selected  for  examination.  The  first  concerns  attributes  of 
existing  procedures.  Questions  regarding  the  nomenclature,  form, 
source,  and  extent  of  individual  crew  member  involvement  must  be 
addressed  before  representative  procedures  can  be  chosen  for  more 
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intensive  scrutiny.  The  product  of  this  first  portion  of  the  study  is 
a  collection  of  taxonomies.  Jointly,  they  characterize  the  procedural 
framework  within  which  one  airline's  crews  were  directed  to  operate  in 
early  1976. 

The  second  area  of  SOP  research  deals  with  the  definition,  calcu¬ 
lation,  and  interpretation  of  descriptive  statistics  associated  with  a 
set  of  representative  crew  coordination  procedures  and  behaviors. 
Questions  about  the  appropriateness  of  procedural  imperatives  and  the 
conformance  of  aircrew  behavior  are  discussed.  One  result  is  a  quanti¬ 
tative  assessment  of  ten  crews'  adherence  to  sampled  procedures  in  a 
high  fidelity,  full  mission  simulation  experiment.  Also,  statistical 
models  are  used  to  test  the  distribution  of  enumerated  operator  errors 
and  to  examine  the  empirical  relationships  among  independent  measures 
of  human  error  and  procedural  behavior. 

A  minor  portion  of  the  research  identifies  three  precepts  of 
management  which  are  particularly  appropriate  to  the  cockpit  situation. 
Each  captain's  management  style  is  subjectively  graded  against  these 
precepts,  and  the  qualitative  assessments  are  compared  to  the  pre¬ 
viously  developed  quantitative  yardsticks  of  procedural  compliance 
and  operator  error.  Relationships  among  different  operator  metrics 
are  suggested. 

The  remainder  of  this  dissertation  is  divided  into  five  distinct 
but  interdependent  chapters.  Chapter  II  contains  a  review  of 
related  literature,  descriptions  of  pertinent  on-going  research,  and 
a  detailed  discussion  of  the  experiment  which  provided  essential  data. 
The  third,  fourth,  and  fifth  chapters  respectively  treat  the 
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aforementioned  subdivisions  of  the  investigation-procedural  taxonomies, 
quantitative  assessments  of  compliance,  and  procedural  conformity  as  a 
reflection  of  management  style.  Major  conclusions  and  recommendations 
for  further  research  are  included  in  the  final  chapter. 

A  compilation  of  aviation  acronyms  and  abbreviations  used 
throughout  the  manuscript  is  provided  in  Appendix  A.  A  flow  chart 
depicting  the  various  analytical  efforts  in  the  research  process 
appears  in  Appendix  B. 
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CHAPTER  II 

RELATED  LITERATURE  AND  RESEARCH 

Allusions  to  vaguely  defined  "operating  procedures"  can  be  found 
in  numerous  technical  reports  (e.g.,  23,31,52,68)  and  journal  articles 
(e.g.,  Human  Factors  and  Ergonomics)  under  titles  dealing  with  flight 
simulation,  safety,  decision  making,  and  human  performance.  Refer¬ 
ences  to  specific  aircraft  or  airline  SOPs  are  commonplace  in  crew 
publications  (e.g..  Aerospace  Safety,  Crosscheck,  or  Air  Line  Pilot) 
and  accident  reports,  several  of  which  are  cited  below.  Expository 
treatments  of  procedures  are  usually  limited  to  textbooks  (28,29,48) 
within  the  general  discipline  known  as  human  factors.  All  of  these 
contribute  to  an  appreciation  of  the  purpose  and  status  of  SOPs  in  the 
multifaceted  operating  environment  of  commercial  aviation. 

This  chapter  highlights  material  from  limited  access  as  well  as 
open  sources.  It  is  divided  into  five  sections.  The  first  section 
reviews  evidence  of  procedures  in  the  context  of  airline  system 
failures.  The  second  section  looks  at  nonspecific  man-machine  pro¬ 
cedures  from  a  designer's  point  of  view.  Past  and  present  research 
bearing  potential  implications  about  the  essence  of  SOPs  is  covered 
in  the  next  two  sections.  The  final  section  summarizes  pertinent 
information  from  Ruffell  Smith's  NASA-sponsored  airline  study.  (53) 
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Operating  Procedures  and  Airline  Accidents 

A  1974  review  of  "pilot  error"  accidents  by  the  Lovelace  Founda¬ 
tion  states  that  "Airline  training  and  operational  procedures  must  be 
updated  to  improve  the  reliability  of  the  pilot  as  an  information 
processer. "  (23:55)  The  report  does  not  recommend  modification  of  any 
specific  procedure,  but  readers  are  given  the  unmistakable  impression 
that  existing  SOPs  were  inadequate.  Crew  member  complicity  in  acci¬ 
dents  receives  little  attention,  but  procedural  noncompliance  is  openly 
acknowledged  as  a  fact  of  airline  life.  The  authors,  some  of  whom 
were  and  are  airline  pilots,  admit  the  long-range  adverse  effects 
of  accepting  "out  of  tolerance"  conditions  and  operating  "off-profile." 
(23:50)  They  do  not,  however,  suggest  even  a  cursory  analysis  of  such 
pilot  behavior.  Like  many  other  powerful  lobbies  in  the  aviation 
community,  airline  pilots  would  never  permit  themselves  to  become  the 
scapegoats  for  diverse  deficiencies  in  air  carrier  safety.  Though 
problems  with  procedural  compliance  have  been  recognized  for  some 
time,  political  and  economic  considerations  continue  to  impede  the 
collection  and  publication  of  hard  data  to  support  the  insider's  wis¬ 
dom  about  noncompliance. 

In  a  more  recent  (1979)  report  for  the  Federal  Aviation  Adminis¬ 
tration  (FAA)  two  Battelle  researchers  analyze  air  carrier  accident 
history  from  1964  through  1976.  They  conclude  that  the  percentage 
of  accidents  attributable  to  human  error  (pilot,  mechanic,  air  traffic 
controller,  etc.)  has  been  increasing.  (8:4-8)  In  giving  special 
attention  to  pilot  complicity,  they  point  out  that  "The  gravity  of 
this  problem  area  is  accentuated  by  the  fact  that  these  human  errors 
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are  occurring  in  spite  of  the  elaborate  mechanical  and  procedural 
systems  now  in  place  to  mitigate  these  errors."  (8:4-9)  Support  for 
their  assertion  can  be  found  throughout  airline  accident  reports,  but 
the  authors  contention  that  existing  SOPs  are  "well  written"  (8:4-6) 
seems  to  be  pure  conjecture.  Acceptance  of  a  procedural  ideal  is  not 
equivalent  to  acceptance  of  existing  procedures  as  ideal.  Standardized 
aircrew  directives  as  well  as  the  operators  for  whom  they  are  designed 
must  both  be  viewed  critically. 

The  Battelle  analysts'  opinion  that  SOPs  constitute  a  de  facto 
contribution  to  aviation  safety  appears  to  be  shared  by  the  National 
Transportation  Safety  Board  (NTSB) .  The  NSTB  regularly  cites  failure 
to  comply  with  explicit  SOPs  and  failure  to  challenge  noncompliance 
as  casual  factors  in  airline  accidents.  Quotes  from  three  representa¬ 
tive  Aircraft  Accident  Reports  (AARs)  disclose  the  Board's  bias  in 
favor  of  published  procedures.  The  first  citation  concerns  the  crash 
of  a  United  Airlines'  DC-8  in  the  mountains  northeast  of  Salt  Lake 
City.  The  Probable  Cause  section  of  the  report  reads  as  follows: 

The  National  Transportation  Safety  Board  determines  that 
the  probable  cause  of  this  accident  was  the  approach 
controller's  issuance  and  the  flightcrew's  acceptance 
of  an  incomplete  and  ambiguous  holding  clearance  in 
combination  with  the  flightcrew's  failure  to  adhere  to 
prescribed  irapairment-of-coramunications  procedures  and 
prescribed  holding  procedures.  The  controller's  and 
flightcrew's  actions  are  attributed  to  probable  habits  of 
imprecise  communication  and  imprecise  adherence  to  pro¬ 
cedures  developed  through  years  of  exposure  to  opera¬ 
tions  in  a  radar  environment. 

Contributing  to  the  accident  was  the  failure  of 
the  aircraft's  No.  1  electrical  system  for  unknown 
reasons.  (45:34) 
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In  reviewing  the  circumstances  surrounding  the  crash  of  a  National 
Airlines'  Boeing  727  in  Escambia  Bay  off  Pensacola,  Florida,  the  NSTB 
again  refers  to  both  aircrew  and  Air  Traffic  Control  (ATC)  procedures. 
For  example. 

National  Airlines’  B-727  procedures  do  not  recom¬ 
mend  that  the  flight  crew  insert  the  MDA  [minimum 
descent  altitude]  into  the  altitude  alert  system. 

(38:19) 

In  summary,  the  Safety  Board  concludes  that  the 
ATC  procedures  affected  the  conduct  of  the  approach, 
and,  therefore,  contributed  to  the  chain  of  events 
which  led  to  the  accident.  Although  the  controller 
had  placed  the  aircraft  in  a  position  from  which 
the  approach  could  have  been  completed  safely,  he  also 
had  placed  it  in  a  position  where  the  captain  had  to 
alter  the  timing  of  his  checklist  procedures  in  order 
to  configure  his  aircraft  more  rapidly  than  usual. 

While  the  controller's  handling  of  the  flight  did  not 
place  the  aircraft  in  a  dangerous  position,  his  non¬ 
standard  procedures  made  the  approach  more  difficult 
for  the  crew  to  accomplish.  (38:33) 

Finally,  a  large  portion  of  the  report  about  an  Allegheny  Air¬ 
lines'  BAC  1-11  mishap  at  Rochester,  New  York  is  devoted  to  direct 
quotation  of  crew  member  SOPs  from  airline  manuals.  Three  of  the 
Board's  15  Findings  are  notable  for  their  dependence  upon  procedural 
imperatives: 

7.  The  crew  did  not  comply  with  checklist  procedures 
during  the  approach  and  landing  in  that  no  callouts 
were  made  and  cockpit  Instruments  were  not  monitored. 

8.  The  crew  failed  to  comply  with  recommended  ap¬ 
proach  and  landing  airspeeds. 

9.  The  approach  was  not  made  according  to  prescribed 
procedures  and  was  not  stabilized.  (34:27) 

These  Safety  Board  comments  indicate  a  degree  of  blind  faith  in 
SOPs  which  is  not  always  warranted.  The  AAR  for  American  Airlines’ 
Flight  191,  which  crashed  shortly  after  takeoff  from  O'Hare  Airport, 


reveals  that  the  crew  did  act  in  accordance  with  the  existing  engine- 
out  procedure;  but  the  procedure  did  not  account  for  the  magnitude  of 
the  failure  that  had  occurred.  (35:53-54)  This  kind  of  knowledge  is 
probably  not  conducive  to  passenger  or  pilot  confidence  in  SOPs,  and 
it  could  be  used  as  rationale  for  selective  noncompliance.  Regardless, 
two  essential  questions  remain  unanswered.  How  meticulous  is  the  aver¬ 
age  crew  concerning  routine  procedures?  Are  the  prescribed  procedures 
appropriate  in  the  typical  operating  environment?  No  documents  in 
the  open  literature  properly  address  these  questions.  This  research 
is  an  attempt  to  partially  fill  that  void. 

Procedures  in  General 

As  man-machine  systems  have  become  more  complex,  ergonomists  have 
been  periodically  redefining  and  enriching  the  concept  of  a  procedure. 
In  the  early  1960s  Kinkade  and  Wheaton  stated  that  "A  procedure  is  a 
step-by-step  series  of  activities  involving  no  special  skill  require¬ 
ments."  (21:676)  Although  that  simplistic  view  would  likely  be 
rejected  by  many  highly  skilled,  highly  trained  operators  (including 
most  pilots),  it  may  indicate  a  basic  human  difficulty  in  appreciat¬ 
ing  someone  else*  s  tasks  and  procedures .  Weiner  observes  that  the 
lack  of  empathy  between  airline  pilots  and  air  traffic  controllers 
can  be  especially  acute.  (66:174)  The  same  difficulty  might  account 
for  coordination  problems  among  particular  crew  members.  The  actions 
of  tyrannical  captains,  reticent  copilots,  and  other  widely  recog¬ 
nized  airborne  personality  types  could  be  magnified  by  an  incomplete 
comprehension  of  the  procedural  demands  on  other  crew  members. 
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Elevation  of  the  status  of  procedures  and  intuition  about  their 
character  advanced  significantly  under  the  mid  1960s  tutelage  of 
Meister  and  Rabideau.  (29)  They  never  synthesize  a  concise  definition 
of  procedures,  but  they  categorize  and  describe  identifiable  types. 

In  particular,  they  distinguish  among  individual  operator,  crew,  and 
system  procedures  (the  first  two  being  more  specific  than  the  last) 
as  well  as  between  operation  and  maintenance  categories.  The  authors 
emphasize  "the  need  for  the  human  engineer  to  pay  great  attention  to 
their  [procedures]  format,  their  information-content,  and  how  they  are 
developed."  (29:109)  Seven  attributes  of  a  well  designed  procedure 
are  listed,  including  the  use  of  a  tabular  checklist  format.  The 
steps  in  each  checklist  are  to  meet  the  following  sequential  require¬ 
ments: 

1.  Time  at  which  the  step  is  to  be  performed. 

2.  If  the  task  as  a  whole  requires  more  than  one 
operator,  the  particular  operator  (by  job  title)  who 
will  perform  the  step. 

3.  The  stimulus  for  the  operator  to  initiate  a  pro¬ 
cedural  step. 

4.  The  operator  response  required  (in  terms  of  monitor¬ 
ing,  deciding,  manipulating). 

5.  Required  communications. 

6.  Feedback  display  or  communication.  (29:109) 

Few,  if  any,  cockpit  SOPs  satisfy  all  the  criteria  of  Meister  and 
Rabideau;  though  some  would  certainly  appear  more  acceptable  than 
others.  Their  criteria  are  not  the  only  meaningful  measures  of  worth, 
however.  The  regularity  of  operator  commitment  to  families  of  SOPs 
and  the  nature  of  nonconforming  behavior  can  be  important  indicators 
of  both  procedural  utility  and  personal  style. 


In  a  later  text  Meister  (28)  differentiates  a  procedural  check¬ 
list  (describing  actions  to  be  performed)  from  an  evaluative  checklist 
(describing  a  desired  quality) .  His  commitment  to  the  checklist 
format  for  SOPs  is  so  strong  that  he  never  mentions  other  forms.  He 
does  observe  that  seemingly  small  systems  may  have  more  procedures 
than  an  analyst  can  thoroughly  review — a  fact  clearly  illustrated  in 
this  study.  Consequently,  he  recommends  examination  of  only  those 
SOPs  necessary  for  accomplishment  of  major  system  objectives  and  those 
whose  incorrect  performance  is  "likely  to  lead  to  serious  equipment, 
personnel,  or  system  consequences."  (28:208)  One  shortcoming  of  this 
advice  is  that  in  airline  operations,  at  least,  numerous  SOPs  and 
many  forms  of  noncompliance  have  been  associated  with  disaster.  (39, 
41,43,44) 

Laboratory  Research 

A  1972  review  of  man-machine  experiments  by  H.  M.  Parsons  views 
procedures  more  as  they  really  are  than  as  they  ought  to  be.  His 
idea  of  "self-procedurization"  (48:547),  which  includes  modification 
of  published  procedures  as  well  as  development  of  new  ones,  recog¬ 
nizes  a  fundamental  reality  of  human  behavior.  People  adopt  and  rely 
on  behavior  patterns  without  much  conscious  effort.  Manifestations 
of  this  phenomenon  in  airline  cockpits  have  previously  been  implied 
but  not  documented.  Possible  evidence  of  their  existence  occurs  in 
the  data  for  this  research. 

Parsons  also  comments  on  a  major  issue  of  this  investigation.  In 


writing  on  the  diversity  of  procedural  variables  he  says,  "If  they 


16 

tend  to  constitute  a  grab-bag,  this  may  be  so  partly  because  there 
exists  in  the  human  engineering  literature  no  taxonomy  of  procedures, 
nor  even  much  of  an  attempt  to  create  one."  (48:208)  Eight  years 
after  that  statement  was  published,  it  still  applies  with  full  force. 
Thus,  this  author  will  take  the  opportunity  to  create  not  one  but 
several  situation  specific  taxonomies. 

Parsons'  recapitulation  of  20  years  (approximately  1948-1967)  of 
human-machine  systems  research  forms  an  appropriate  backdrop  for  more 
recent  studies.  Papers  presented  at  the  International  Symposium  on 
Monitoring  Behavior  and  Supervisory  Control  in  1976  focus  on  the  devel¬ 
opment  of  internal  models  of  human  behavior.  (32,50,57,63)  One  inter¬ 
pretation  of  a  procedure  is  that  it  constitutes  a  major  portion  of  the 
external  input  to  a  corresponding  internal  model.  A  procedural 
ideologue  might  go  so  far  as  to  propose  that  a  properly  designed  SOP 
could  be  wholly  embodied,  without  appendage,  into  an  internal  model.  A 
more  plausible  case  can  be  made  for  the  inclusion  of  additional  inputs, 
internal  as  well  as  external,  which  combine  to  form  an  internal  model 
and  which  are  reflected  in  the  concept  of  self-procedurization. 

Many  participants  in  the  aforementioned  symposium  helped  organize 
a  workshop  on  mental  workload  the  following  year.  There,  Johannsen, 
Moray,  et  al.,  suggested  the  need  for  long  duration,  behaviorally  com¬ 
plex  experiments.  (20:110)  The  Ruff ell  Smith  experiment  is  already 
in  the  vanguard  of  such  research.  In  addition,  Johannsen  crystalized 
the  notion  that  procedures  form  one  portion  of  the  mental  load  placed 
on  skilled  operators;  he  describes  the  environment  and  the  situation 
as  the  other  two,  (19:4)  At  present  no  objective  scales  are  available 
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for  "weighing"  any  portion  of  mental  load.  Simple  enumeration  and 
categorization  offer  the  best  subjective  indication  of  an  operator's 
procedural  burden.  On-going  research  gives  some  hope  for  objective 
measures  and  new  insights. 

Research  in  Progress 

Theories  now  being  developed  by  Funk  suggest  that  a  methodology 
for  objectively  measuring  procedural  load  in  any  man-machine  system 
may  soon  be  realized.  (15)  Numerous  other  researchers  sponsored  by 
the  military  departments  are  engaged  in  studies  which  could  influence 
the  nature  of  procedures  taught  to  the  next  generation  of  combat 
aviators.  Burks,  Engler,  and  Sears  are  using  a  computer  simulation  to 
try  to  identify  control  strategies  [procedures]  in  a  one-dimensional 
tracking  task.  (7:22-23)  Damos  is  doing  a  comparison  of  multiple-task 
performance  using  natural  and  imposed  strategies  to  evaluate  differ¬ 
ences  in  cognitive  style.  (10:26-27)  Models  which  distinguish  between 
the  mental  processes  of  novice  and  skilled  operators  are  being  investi¬ 
gated  by  Dreyfus  and  Dreyfus  (12:28-29)  They  have  hypothesized  that 
novices  rely  heavily  on  predetermined  procedures  while  experts  improvise 
their  own. 

Perceptronics,  Incorporated  and  the  Canyon  Research  Group  are 
conducting  studies  specifically  concerned  with  aircrew  behavior.  The 
first  is  an  attempt  to  create  taxonomies  of  emergency  situations  and 
the  decision  processes  that  pilots  use  to  resolve  those  situations. 
(25:48-49)  That  study  is  an  off-shoot  of  earlier  research  in  self- 
procedurization  by  the  Air  Force  Human  Resources  Laboratory.  (59) 


Canyon  has  tri-service  sponsorship  of  its  effort  to  facilitate  compari¬ 
son  of  disparate  human  factors  research  by  establishing  standardized 
protocols  for  measurement  of  crew  performance.  (64:102-103)  Taken 
together,  the  results  of  these  studies  could  significantly  alter  the 
role  of  combat  pilots  and  the  characteristics  of  their  SOPs.  However, 
any  impact  on  air  carrier  pilots  would  probably  be  negligible.  For 
research  involving  airline  procedures  one  must  turn  to  Ruff ell  Smith's 
work. 

The  Ruffell  Smith  Experiment 

As  early  as  1974  Dr.  H.  P.  Ruffell  Smith,  a  noted  British  physi¬ 
cian,  ergonomist,  and  pilot,  proposed  a  full  mission  flight  simulation 
experiment  to  study  the  performance  of  fully  qualified  airline  crews 
under  varying  conditions  of  workload.  (53:1)  The  proposal  garnered 
the  support  of  the  National  Research  Council  and  NASA.  Intensive 
preparations  began  in  July  1975;  data  collection  took  place  early  in 
1976. 

An  unidentified  major  air  carrier  agreed  to  provide  a  state-of- 
the-art  simulation,  volunteer  crews,  and  technical  assistance.  Because 
of  the  company's  training  schedule,  each  simulator  session  was 
constrained  to  require  no  more  than  four  and  one-half  hours.  The 
simulator  was  a  Singer-Link  platform  with  six  degrees  of  motion  free¬ 
dom.  The  cockpit  was  that  of  a  Boeing  747  configured  identically  to 
company  aircraft.  It  accommodated  the  usual  three-person  crew 
(captain,  copilot,  and  flight  engineer)  plus  two  observers  (experi¬ 
menters),  a  simulator  operator/ traff ic  controller  (company  training 
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captain)  and  an  audio  coordinator.  The  latter  managed  playback  of  the 
ATC  tapes  that  were  used  throughout  the  scenario  (12  for  each  of  the 
two  mission  segments)  to  enhance  realism.  Since  computer  generated 
imagery  was  not  installed,  video  films  of  the  landing  runway  were 
used. 

a.  Scenario 

A  full  mission  scenario  was  assembled  by  Dr.  Ruffell  Smith,  Mr. 
George  Cooper,  former  Chief  Test  Pilot  of  NASA-Ames,  and  a  senior 
flight  instructor  from  the  participating  airline.  The  scenario  was 
predicated  upon  charter  service  from  Dulles  Airport  near  Baltimore  to 
Heathrow  Airport  (London)  with  a  30-minute  intermediate  stop  at 
Kennedy  Airport  (JFK,  New  York)  for  fuel  and  cargo.  The  first  segment 
of  the  mission  was  intended  to  place  relatively  low  workload  on  the 
crews;  the  load  during  the  second  segment  was  to  be  much  higher.  Crews 
were  not  aware  of  the  workload  differences  before  their  flights. 

One  of  the  three  flight  data  computers  (for  autopilot  and  flight 
director  operation)  was  disabled  throughout  the  mission;  otherwise, 
normal  mechanical  redundancies  were  operative.  There  were  no  pre¬ 
programmed  equipment  failures  during  the  Dulles-JFK  leg.  Since  most 
of  the  18  test  crews  were  unfamiliar  with  the  routes/airports  involved 
and  since  they  were  not  given  avigation  information  until  arrival  for 
the  mission,  this  writer  believes  that  the  actual  mental  load  for  the 
first  leg  may  have  been  considerably  higher  than  intended.  Even  so, 
only  published  normal  operating  procedures  were  needed  during  that 
segment.  ATC  vectoring  around  a  small  area  of  thunderstorms  was 
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included  as  a  routine  occurrence,  but  it  may  also  have  produced  an 
unintended  increase  in  workload. 

Because  the  Dulles-JFK  leg  of  the  scenario  tried  to  recreate  a 
typical  operating  environment,  the  crew  activity  associated  with  it  is 
thought  to  be  representative  of  habitual  behavior.  In  contrast,  the 
second  segment  incorporated  two  mechanical  failures,  one  of  which 
necessitated  an  engine  shutdown,  diversion  to  an  alternate  airport 
(return  to  JFK),  fuel  dumping,  holding,  repeated  distractions  by 
dispatch/cabin  crew  personnel  and  a  heavyweight  landing  on  a  slippery 
runway  with  a  strong  crosswind.  Although  crew  behavior  in  unantici¬ 
pated,  stressful  situations  deserves  careful  consideration,  it  is  not 
of  primary  interest  here. 

Crew  adherence  to  standard  operating  procedures  is  of  concern  in 
this  study.  Cockpit  management,  task  pacing,  and  control  of  aircraft 
parameters  caused  the  length  of  time  spent  in  the  simulator  to  vary 
slightly.  For  example,  some  pilots  slowed  to  approach  speed  further 
from  their  destination  than  others;  but  the  same  basic  SOPs  were 
appropriate  for  every  experimental  run.  Explicit  procedural  directives 
were  adequate  for  all  situations.  Imaginative  or  creative  responses  to 
scenario  developments  were  not  required.  Nevertheless,  cases  of  non- 
compliance  with  procedure,  perceptual  uncertainty,  and  judgmental 
error  resiilted  in  different  procedural  sequences  and  loads  across  the 


crews. 
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b.  Simulation  Fidelity 

Ruffell  Smith’s  goal  was  to  simulate  the  operating  environment  in 
sufficient  detail  to  evoke  the  same  operator  behavior  that  occurs  in 
actual  flight.  Determining  the  degree  of  contextual  detail  necessary 
for  a  "full  mission"  simulation  was  then  and  is  now  more  of  an  art 
than  a  science.  Although  all  of  the  participants  in  the  experiment 
affirmed  the  prevailing  realism  of  the  scenario  as  presented,  several 
deviations  from  verisimilitude  are  detectable. 

To  begin  with,  the  external  visual  cues  were  not  of  high  quality. 
There  were  no  external  cues  for  taxi  or  takeoff.  The  landing  runway 
films  lacked  the  scope  and  flexibility  which  computer  generated  images 
have  today.  The  research  simply  had  to  proceed  with  the  simulator 
technology  on  hand.  That  constraint  also  accounts  for  the  occasionally 
unsatisfactory  functioning  of  the  inertial  navigation  system  (INS)  and 
the  one  serviceable  autopilot,  discrepancies  which  were  noticed  by  the 
crews.  These  deficiencies  seem  to  have  had  only  a  tangential  effect 
on  individual  and  crew  behaviors. 

An  aspect  of  the  simulation  which  did  temporarily  distract  or 
confuse  several  crew  members  centered  upon  the  background  audio 
recordings.  In  the  first  segment  one  tape  was  too  short  for  its  phase 
of  the  mission  so  the  technician  continuously  replayed  it.  Two  other 
tapes  contained  operating  information  which  was  different  from  that 
issued  to  the  crews  by  the  simulator  controller,  thus  necessitating 
some  clarification.  Finally,  the  recorded  voices  of  air  traffic 
controllers  changed  from  tape  to  tape,  but  the  live  voice  of  the 
simulator  controller  was  always  the  same  (and  at  a  different  pitch 
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and  volume  from  tnose  on  tape).  Moreover,  the  quality  of  simulator 
controller  transmissions  was  not  on  a  par  with  that  usually  emanating 
from  professional  air  traffic  controllers.  These  facts  may  have  influ¬ 
ences  crew  behavior  with  respect  to  extra-cockpit  radio  communications. 
For  example,  constant  omission  of  the  "Heavy"  suffix  to  the  aircraft 
call  sign  by  the  controller  seems  to  have  induced  some  crews  to 
respond  similarly.  The  effect  on  other  types  of  crew  behavior  is 
unknown. 

Two  other  features  of  the  experiment  could  have  persuaded  crew 
members  to  depart  from  the  physiological  and  psychological  normalcy 
of  an  actual  flight  mission.  First,  a  single  electrocardiogram  (ECG) 
electrode  was  applied  to  each  subject  throughout  the  mission.  Second, 
because  they  were  in  a  simulator  usually  reserved  for  emergency  actions 
training  and  evaluation,  crew  members  were  initially  sensitive  to  and 
suspicious  of  potential  mechanical  problems.  Beyond  a  very  brief 
period  of  adjustment  to  these  conditions,  neither  evoked  any  negative 
comments  from  the  subjects.  One  might  reason  that  given  the  obvious 
scrutiny,  crews  would  likely  be  on  their  "best  (procedural)  behavior." 
The  reader  may  draw  his  own  conclusions  on  this  issue  after  reviewing 
the  relevant  data. 

c.  Subjects 

Most  of  the  participants  were  assigned  to  a  single  aircrew  domicle. 
The  one  captain  who  came  from  a  different  domicile  is  subjectively 
ranked  between  the  extremes  of  managerial  performance.  The  number  of 
errors  attributed  to  him  and  his  crew  combined  with  their  aggregate 
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adherence  to  procedures  rendered  their  overall  performance  genuinely 
"average."  Some  of  the  subjects  had  flown  together  before;  others 
had  not.  Some  of  the  highest  and  some  of  the  lowest  error  counts  were 
generated  by  crew  members  flying  together  for  the  first  time.  In  the 
pragmatic  world  of  standard  operating  procedures  and  forceful  labor 
unions,  crew  composition  is  not  supposed  to  be  a  factor  in  procedural 
conformance  or  in  operational  safety. 

The  experiment  used  18  crews  for  comparative  data  purposes.  Since 
captains  and  copilots  typically  alternate  mission  responsibilities  as 
the  "pilot  flying"  (PF)  and  "pilot  not  flying"  (PNF)  for  whole  flight 
segments,  the  experimenters  permitted  each  crew  to  determine  who  would 
"fly"  which  leg.  Thirteen  of  the  eighteen  captains  were  the  PF  for 
the  Dulles-JFK  segment.  This  could  be  indicative  of  many  captains' 
desire  to  establish  their  prerogative  at  the  earliest  opportunity,  but 
there  is  no  data  to  support  such  speculation. 

The  subjects  were  nearly  unanimous  in  complimenting  the  design 
and  execution  of  the  scenario.  Some  wrote  letters  to  the  research 
team  expressing  appreciation  for  the  opportunity  to  participate  and 
concern  about  the  general  state  of  crew  performance.  Several  volun¬ 
teered  to  participate  in  any  extensions  of  the  research. 

d.  Data 

Since  the  technical  quality  of  the  data  collected  and  the 
experimentum  personae  were  not  uniform  for  all  eighteen  simulator  runs, 
it  has  become  necessary  to  eliminate  some  trials  from  the  current 
investigation.  Ten  runs  have  the  same  pair  of  observers  and  usable 
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audio  data  throughout  have  been  selected  for  detailed  procedural  analy¬ 
sis.  The  characteristics  of  the  ten  crews  and  the  range  of  observed 
behaviors  compare  favorably  with  those  of  the  total  experimental  popu¬ 
lation.  Historical  information  for  the  sample  and  the  total  population 
is  summarized  in  Table  2. 

The  ranges  on  age  and  years  of  flying  experience  for  captains  in 
the  experimental  population  are  relatively  small,  54  to  59  and  32  go  41 
respectively.  These  numbers  represent  considerable  seniority  with 
the  airline,  familiarity  with  company  policies,  and  exposure  to  diverse 
operational  conditions.  These  men  are  at  the  top  of  the  company's 
pilot  ladder. 

The  copilots,  though  younger  (average  age  43.4  versus  55.8  for 
captains),  all  have  at  least  16  years  of  flying  experience.  Some  have 
enough  seniority  to  soon  qualify  for  upgrade  as  captains  of  the 
company's  smaller  planes.  They  might  reasonably  be  anticipating  the 
responsibilities  and  rewards  of  captaincy.  Other  copilots  have  little 
experience  in  the  B-747  and  may  be  looking  to  their  cohorts  for 
operational  acumen. 

The  flight  engineers  are  among  the  last  of  an  old  order.  They 
are  career  flight  engineers  as  opposed  to  pilots  passing  through  the 
second  officer  rank  on  the  way  to  becoming  a  first  officer  (copilot). 
Since  career  engineers  are  no  longer  hired  by  the  airline,  the  patterns 
of  crew  member  experience  are  now  changing.  In  the  near  future  the 
flight  engineer  will  usually  be  the  youngest  member  of  any  crew.  In 
the  experimental  population  the  average  age  of  flight  engineers  is 
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slightly  less  than  that  of  captains  (roughly  three  years)  and  decidedly 
greater  than  that  of  copilots  (about  eight  years) .  In  several  cases 
the  engineer  has  more  B-747  flying  time  than  either  one  of  the 
pilots.  However,  the  effect  of  different  combinations  of  experience 
is  impossible  to  isolate  in  the  data. 

Three  separate  modes  of  data  collection  were  used  in  the  experi¬ 
ment:  frequency  modulated  (FM)  tape,  hand  written  documents,  and 
computer  printout  of  aircraft  (i.e.,  simulator)  parameters.  In  the 
order  of  recording,  the  seven  tracks  of  FM  tape  contained  the  follow¬ 
ing  information:  (1)  captain’s  ECG,  (2)  copilot's  ECG,  (3)  flight 
engineer's  ECG,  (4)  pilot  observer  commentary,  (5)  engineer  observer 
commentary,  (6)  intra-cockpit  communications,  and  (7)  extra-cockpit 
communications.  The  four  audio  tracks  are  the  primary  sources  of  data 
on  procedural  behavior. 

At  the  end  of  every  mission  the  observers  and  the  training 
captain  transcribed  their  recollections  and  impressions  in  longhand  on 
protocols.  Biographical  data  such  as  that  shown  in  Table  2  were 
logged  prior  to  each  mission.  Takeoff  computation  sheets,  fuel  consump¬ 
tion  logs,  and  any  other  pieces  of  crew  paperwork  completed  the  pencil 
and  paper  data. 

A  high-speed  line  printer  connected  to  the  computer  which  was 
driving  the  aircraft  simulator  provided  the  third  data  collection  mode. 
Seventeen  different  aircraft  parameters  were  sampled;  many  others 
might  have  been  useful.  Output  was  generated  from  the  application  of 
takeoff  thrust  until  the  end  of  the  landing  rollout.  Data  were  printed 
once  every  second  when  the  aircraft  was  below  1800  feet  and  once 
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every  five  seconds  above  that  altitude.  These  data  help  define  the 
environmental  state  of  the  system  at  particular  points  in  time,  but 
they  contribute  little  to  an  appreciation  of  the  operator  behavior 
which  produced  the  various  states. 

e.  Purpose  and  Problems 

Ruffell  Smith  has  written  that  the  study  was  "a  specific  attempt 
to  investigate  the  kind  and  number  of  errors  [by  crews]  and  how  these 
related  to  overall  workload  and  arousal."  (53:14)  Unfortunately, 
neither  precise  definitions  nor  bibliographic  references  are  offered 
for  such  essential  concepts  as  workload,  arousal,  and  error.  However, 
in  view  of  the  numerous  unresolved  issues  in  conceptualizing  and 
measuring  pilot  workload  presented  by  Gartner  and  Murphy  (17)  this  is 
understandable.  While  contemporary  usage  of  "workload"  permits  a  basic 
low/high  distinction  between  the  two  mission  legs,  fluctuations  of 
load  within  each  leg  appear  significant.  No  specific  measure  of 
workload  is  ever  mentioned. 

Likewise,  the  notion  of  arousal  is  treated  untuitively.  It  seems 
to  be  somehow  linked  to  workload  in  the  experimenter's  mind,  but  the 
relationship  is  never  explained.  Different  types  of  workload  and 
arousal  (e.g.,  mental  versus  physical)  are  not  discussed,  but  pure 
heart  rate  is  identified  as  the  sole  measure  of  arousal. 

Ruffell  Smith's  explanatory  neglect  is  perhaps  most  severe  regard¬ 
ing  the  concept  of  error.  Performance  tolerances  or  scales  are  rarely 
provided.  The  subjective  judgment  of  the  experimenters  appears  to  be 
the  usual  yardstick.  According  to  the  final  report,  "Special  notice 


was  taken  of  errors  in  procedures  (e.g.,  the  use  of  checklists)  and  of 
specific  errors  (e.g.,  mistakes  in  setting  up  navigation  and  communi¬ 
cation  frequencies)."  (53:12)  However,  his  nine  error  categories  do 
not  clearly  make  such  distinctions.  A  summary  of  the  errors  (by 
category  and  by  crew)  that  the  author  identified  during  the  Dulles-JFK 
segment  is  shown  in  Table  3. 

Ruffell  Smith's  confession  that  the  errors  are  "arbitrarily 
classified"  (53:14)  reveals  nothing  about  the  underlying  purpose  of 
his  scheme.  Errors  are  attributed  to  an  entire  crew  regardless  of  how 
many  people  have  the  opportunity,  knowledge,  and  ability  to  either 
prevent  or  rectify  a  given  fault.  For  example,  the  stated  distinction 
between  a  flight  engineer's  error  in  improperly  emptying  a  fuel  tank 
and  an  entire  crew's  concurrence  in  loading  with  an  improper  flap 
setting  is  that  the  former  is  considered  to  be  Systems  Operation  while 
the  latter  is  designated  as  a  Tactical  Decision. 

Supposed  distinctions  between  Flying  errors  and  Flying  Skill 
errors  exacerbate  a  reader's  frustration.  They  certainly  do  not  aid 
in  the  recognition  of  crew  member  behavior  patterns.  According  to  the 
report.  Flying  errors  arise  when  a  pilot  is  judged  to  have  successfully 
controlled  his  aircraft  to  the  wrong  parameters,  e.g.,  maintaining  270 
knots  below  10,000  feet.  Flying  Skill  errors  pertain  to  a  pilot's 
presumed  inability  to  control  his  aircraft  to  the  parameters  he  desires. 
Failure  to  stay  within  ten  knots  of  desired  speed  during  an  approach 
and  "rough"  handling  of  the  throttles  on  final  (even  if  the  speed 
criterion  is  satisfied)  are  examples  of  errors  in  Flying  Skill. 


ERRORS  REPORTED  BY  RUFFELL  SMITH 
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Distinguishing  between  these  categories  requires  an  estimation  of 
pilot  intent  which  the  data  cannot  support. 

f.  Conclusions 

According  to  Dr.  Ruff ell  Smith  crews  committed  more  errors  per 
unit  time  during  the  second  or  "high"  workload  segment  than  during 
the  "low."  This  result  is  intuitively  appealing,  but  it  does  not  pro¬ 
vide  any  information  about  the  nature  or  distribution  of  the  errors 
observed  in  either  segment.  Although  experimental  interest  was  con¬ 
centrated  on  the  second  leg  of  the  mission,  few  conclusive  quantita¬ 
tive  results  were  captured  there. 

The  researchers  were  especially  interested  in  establishing 
statistically  significant  physiological  or  historical  predictors  of 
crew  performance  during  the  second  leg.  The  most  significant  relation¬ 
ship  found  was  that  changes  in  the  status  quo  of  the  aircraft  produced 
a  higher  percentage  rise  in  heart  rate  for  the  PF  than  for  the  PNF 
regardless  of  crew  qualification  (i.e.,  captain  or  copilot).  (53:30) 

The  analysis  of  variance  relationships  between  total  number  of  errors 
in  the  first  segment  and  days  since  last  flight  were  significant  for 
captains  and  flight  engineers,  but  not  for  copilots.  (53:30)  No  such 
relationships  were  significant  for  the  second  segment.  Similarly, 
first  leg  error  totals  were  not  a  good  predictor  of  second  leg  error 
counts.  Relationships  between  by-operator  errors  and  specific  by¬ 
operator  behaviors  were  not  addressed. 


g.  Comments 

It  is  particularly  interesting  to  note  that  Ruffell  Smith  displays 
errors  by  crew  rather  than  by  individual  operator  or  combinations 
thereof.  Examination  of  his  protocols  and  working  papers  reveals  that 
sometimes  errors  are  attributed  to  particular  crew  members;  other  times 
they  are  not.  Multiple  reviews  of  the  recorded  data  have  convinced 
this  writer  of  the  need  to  reclassify  errors  according  to  the  crew 
member(s)  responsible.  This,  in  turn,  facilitates  a  comparison  of 
individual  procedural  behaviors  (yet  to  be  defined)  with  individual 
and  group  errors  previously  identified. 

The  efforts  of  Dr.  Ruffell  Smith  and  his  associates  were  not 
without  some  technical  and  analytical  shortcomings.  However,  these 
should  not  be  overemphasized.  The  investigation  was  "breaking  new 
ground"  in  applied  aircrew  research.  No  study  of  comparable  magnitude 
and  detail  has  ever  been  released  to  the  public.  The  voluminous  data 
could  easily  support  future  analyses  in  small  group  processes,  verbal 
communications,  or  manual  control.  Further  analysis  of  procedural 


compliance,  especially  in  the  second  segment  of  the  scenario,  should 
be  initiated  as  soon  as  possible. 


CHAPTER  III 


ENUMERATION  AND  CLASSIFICATION  OF 
AIRCREW  PROCEDURES 

In  the  final  paragraph  of  his  NASA  report  Dr.  Ruff ell  Smith 
asserts  that. 

The  same  techniques  [of  full  mission  simulation] 
might  be  beneficial  in  developing  and  validating 
standard  operating  procedures  to  achieve  optimum 
integration  of  flightcrews  and  to  avoid  conflicting 
instructions  and  activities.  (53:35) 

In  this  statement  he  seems  to  align  himself  with  those  who  believe  that 
established  procedures  are  completely  acceptable  and  that  new  pro¬ 
cedures  are  more  deserving  of  analysis  than  present  ones.  Yet,  his  own 
data  reveal  problems  of  crew  integration  and  conflicting  activity  which 
relate  directly  to  noncorap liance  with  explicit  SOPs.  A  study  of  exist¬ 
ing  procedural  imperatives  and  crew  behavior  ought  to  begin  precisely 
where  the  published  report  stops.  The  manuals,  charts,  approach 
plates,  data  tapes,  and  protocols  from  the  original  experiment  are 
available  to  extend  the  previous  research.  The  FAA  and  company  pre¬ 
scribed  SOPs  applicable  to  routine  operations  such  as  the  first  seg¬ 
ment  of  the  experimental  scenario  constitute  a  norm  against  which  the 
subjects'  behaviors  can  be  judged. 

The  present  chapter  clarifies  the  procedural  milieu  of  the 
research.  It  begins  with  examples  of  semantic  difficulties  which  tend 


to  obscure  recognition  of  operating  procedures.  Next,  a  basic  set  of 
SOPs  applicable  to  the  Dulles-JFK  segment  of  the  experimental 
scenario  is  enumerated.  Both  empirical  and  analytical  taxonomies  are 
constructed  using  the  identified  procedures.  Finally,  a  subset  of 
SOPs  associated  with  leadership  and  crew  coordination  is  selected  for 
further  analysis. 

Semantics 

Before  enumerating  or  classifying  a  single  aircrew  operating  pro¬ 
cedure,  a  review  of  the  definition  in  Chapter  I  is  appropriate. 

A  procedure  is  a  symbolic  and  mnemonic  representa¬ 
tion  of  a  set  of  sensory,  cognitive,  and/or  motor 
activities  which,  when  recalled  and  executed  within 
determinable  tolerances,  complete  a  task  as  designed. 

This  conceptualization  is  unlike  any  reference  to  procedure  in  aircrew 
directives.  In  fact,  the  available  crew  documents  do  not  define  a 
generic  procedure  at  all.  [Henceforth,  all  references  to  specific 
aircrew  directives  shall  be  those  of  the  FAA  and  the  unnamed  air 
carrier  involved  in  the  Ruffell  Smith  experiment.]  Still,  the  word 
"procedure"  and  the  concepts  embodied  in  it,  often  masked  by  a  variety 
of  aliases,  appear  throughout  the  literature  of  operational  aviation. 

The  aircraft  operating  manual  (AOM)  and  the  company  operations 
manual  (COM)  are  the  principal,  but  certainly  not  the  only  sources  of 
standard  aircrew  procedures.  Federal  Aviation  Regulations  (FARs) , 
aeronautical  maps  and  charts,  instrument  departure  depictions,  instru¬ 
ment  approach  plates,  airport  directions,  Notices  to  Airmen  (NOTAMs) , 
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crew  handbooks,  and  other  miscellaneous  documents  contribute  to  the 
plethora  of  operating  procedures. 

The  frequency  of  change  and  the  necessity  for  cross  referencing 
vary  with  each  document,  but  collectively  they  require  seemingly 
constant  relearning  and  integration.  The  single  side  area  of  docu¬ 
ments  applicable  to  the  simulated  mission  other  than  the  AOM,  COM,  and 
crew  handbooks  covered  twenty  square  meters.  (53:33-34) 

All  of  the  major  organizations  which  influence  or  regulate  air¬ 
line  operations  and  safety  recognize  the  reality  of  aircrew  procedures, 
but  they  have  no  universally  accepted  definition  which  expounds  the 
meaning  of  the  concept.  For  the  time  being,  the  definition  above  must 
serve  as  a  prototype.  Even  within  the  crew  directives  of  a  single  air 
carrier  the  idea  of  procedure  is  susceptible  to  overuse  and  under¬ 
definition. 

Excerpts  from  the  "Introductions"  to  the  AOM  and  COM  each  indicate 
a  preoccupation  with  the  procedural  mystique.  One  of  the  first  state¬ 
ments  in  the  AOM  is  that  "The  procedures  in  this  manual  include  those 
in  the  FAA  Approved  Flight  Manual  expanded  as  necessary  for  [company] 
operations."  The  COM  is  equally  blunt:  "This  manual  contains  the  poli¬ 
cies,  rules  and  procedures  that  govern  the  operational  activity  of  the 
Operations  department."  Despite  a  whole  chapter  of  definitions  in  the 
COM,  neither  it  nor  the  AOM  explains  what  constitutes  a  procedure. 
Similarly,  neither  publication  points  out  any  of  the  words  which  are 
frequently  used  as  pseudonyms  for  "procedure."  One  basic  reason  may 
be  that  conceptual  understanding  is  lacking. 
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Another  may  be  terminology.  Aviation  jargon  has  clearly  added  to 
the  semantic  confusion  surrounding  SOPs.  A  variety  of  words  used  in 
specific  contexts  connote  prescribed  operator  activity  in  the  same 
sense  that  a  "standard  operating  procedure"  does.  Some  of  the  more 
familiar  terms  are  listed  in  Table  4.  Each  can  have  a  connotation 
akin  to  the  earlier  definition  of  procedure.  An  example  of  a  usage 
having  the  connotation  of  a  behavioral  imperative  accompanies  each 
word. 


TABLE  4 


AVIATION  WORDS  AND  PHRASES  WITH  PROCEDURAL  CONNOTATIONS 


WORD 


COMMON  PHRASE 


callouts 

checklist 

clearance 

directions 

guidelines 

instructions 

limitations 

method 

plan 

policy 

practices 

profile 

process 

regulation 

restriction 

rule 

schedule 

steps 

technique 


altitude  callouts 

the  Pre-Start  Checklist 

an  Air  Traffic  Control  clearance 

directions  for  re-filing  in  flight 

fuel  management  guidelines 

instructions  for  determining  dry  tank  weight 
engine  operating  limitations 
a  method  for  computing  threshold  airspeed 
an  instrument  flight  plan 

company  policy  concerning  copilot  landings 
communications  practices 
a  profile  descent 

the  process  of  troubleshooting  a  malfunction 
the  regulation  on  flight  station  manning 
a  standard  instrument  departure  restriction 
a  rule  for  holding  pattern  entry 
the  optimal  climb  schedule 

the  steps  in  proceeding  direct  to  a  station 
throttle  control  technique  during  an  approach 


Unfortunately,  the  authors  and  editors  of  aircrew  directives, 
including  the  AOM  and  COM,  have  never  been  constrained  to  use  consist¬ 
ent  terminology  with  respect  to  procedures.  In  the  ideal,  a  policy 
should  be  more  abstract  than  a  procedure  which  should  be  more  abstract 
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than  a  technique  which  should  be  more  abstract  than  a  limitation. 
However,  these  words  are  used  almost  interchangeably  in  the  manuals. 

The  COM  has  one  major  section  entitled  Fuel  and  Flight  Planning  Policy 
and  another  named  Instrument  Approach  Procedures,  which  can  be  found 
in  the  same  chapters  with  Aircraft  Loading  Limitations  and  Approach 
and  Landing  Limitations  respectively.  The  level  of  abstraction  is 
essentially  the  same  in  every  section.  Thus,  the  expression  standard 
operating  procedure  (SOP)  has  been  chosen  to  represent  all  words  and 
phrases  with  a  procedural  connotation.  Identification  of  a  finite 
collection  of  SOPs  corresponding  to  the  expected  tasks  in  the  simulated 
mission  sequence  can  now  proceed. 

The  entries  in  Table  4  indicate  the  diversity  of  procedural 
expression.  They  also  suggest  that  procedural  imperatives,  like  the 
tasks  for  which  they  are  designed,  lie  along  a  continuum.  At  one 
extreme  are  simple  single-step  procedures  of  limited  duration  and 
activity.  At  the  other  extreme  are  grand  strategies  which  involve  mul¬ 
tiple  actors,  tasks,  and  sub— procedures . 


Enumeration  of  Operating  Procedures 

Only  those  SOP  applicable  to  routine  flight  operations  are  of 
concern  in  this  research.  In  aviation  phraseology  these  are  the  Normal 
Operating  Procedures  as  opposed  to  the  Abnormal,  Alternate,  Irregular, 
or  Emergency  designations  which  are  also  found  in  crew  publications. 

The  order  of  presentation  does  not  follow  that  of  any  single  crew 
directive.  Rather,  it  is  an  idealized  sequence  based  on  the  events  in 
the  Dulles-JFK  segment  of  Dr.  Ruff ell  Smith's  experimental  scenario 
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(cf.  Chapter  II  for  description). 

After  assembling  the  crew  in  the  flight  dispatching  center,  the 
captain  is  directed  by  the  Predeparture  Planning  Procedure  to  brief 
other  crew  members  and  coordinate  their  activities.  This  SOP  appears 
in  the  Flight  Conduct  chapter  of  the  COM.  It  incorporates  a  lengthy 
list  of  predeparture  activities,  but  it  does  not  specify  a  rigid 
sequence  of  accomplishment  or  the  responsible  operator(s).  Nor  does 
this  procedure  have  a  companion  checklist  for  easy  verification  of 
completed  activities.  Whether  these  characteristics  should  be  classed 
as  defects  remains  open  to  future  investigation.  Each  of  the  numerous 
tasks  of  predeparture  planning  (see  Table  5)  has  its  own  explicit  pro¬ 
cedure  printed  elsewhere.  Most  of  them  tolerate  an  ill-defined  degree 
of  self-procedurization.  Only  the  Engineer's  Aircraft  Preflight  has 
an  abbreviated  checklist  in  addition  to  textual  explanation  of  desired 
behavior. 

The  Predeparture  Planning  Procedure  is  shown  in  toto  because  it 
typifies  a  popular  means  of  providing  normal  operating  strategy. 

Major  milestones  or  considerations  (metatasks)  are  briefly  presented 
without  accompanying  elaboration  on  their  subtasks  or  interdependen¬ 
cies.  Greater  detail  is  provided  with  individual  sub-procedures.  In 
a  complex  multi-dimensional  operating  environment  the  question  of  where 
to  begin  and  end  elaboration  has  no  simple  answer.  Designers  of  SOPs 
face  a  quandry.  Some  level  of  operator  intelligence  must  be  assumed. 
The  problem  is:  how  much?  Requiring  operators  to  regularly  refer  to 
other  parts  of  a  manual  may  be  the  only  concise  way  to  deal  with  mani¬ 
fold  contingencies. 
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TABLE  5 

THE  PREDEPARTURE  PLANNING  PROCEDURE 
Step  Number  Sub-procedures 


1.  Captain’s  briefing. 

2.  Aircraft  preflight. 

3.  Enroute  and  terminal  weather. 

4.  Applicable  NOTAM  [Notice  to  Airmen]  and  mainten¬ 
ance  bulletin  review. 

5.  Flight  plan  and  weight  and  balance  review. 

6.  Track  or  waypoint  verification. 

7.  Verification  of  adequate  fuel  for  dispatch. 

8.  Determination  of  the  limiting  takeoff  gross  weight, 
the  engine  thrust  settings,  and  the  prescribed 
speeds  for  the  anticipated  takeoff  conditions. 

9.  Execution  of  the  dispatch  release. 

10.  Collection  of  required  papers  and  charts. 

11.  Aircraft  maintenance  log  review. 

12.  Restricted  Articles  Loading  Notification  form 
review. 


If  the  necessary  audio  and  video  data  had  been  recorded  [they 
were  not],  analysis  of  crew  conformance  with  the  Predeparture  Planning 
procedure  would  be  a  valuable  study  by  itself.  In  fact,  Predeparture 
Planning  is  representative  of  a  number  of  highly  aggregated  procedures 
which  bear  mightily  on  aircrew  behavior  but  which  generate  physical 
movement  as  well  as  sound.  The  aircraft  ground  and  flight  control 
procedures  are  other  prime  examples.  Their  names  are  basically  the 
same  as  the  operational  phases  of  a  typical  mission  (cf.  Table,  Mission 
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Phase  Taxonomy):  taxi,  takeoff,  departure,  climb,  level  off,  cruise, 
descent,  approach,  landing,  and  parking.  Crew  directives  rely  largely 
on  narrative  or  pictorial  descriptions  of  the  associated  manual  con¬ 
trol  tasks.  The  procedures  appear  deceptively  simple  to  the  uniniti¬ 
ated.  One  reason  for  this  is  that  they  tacitly  assume  adherance  to  a 
multitude  of  sub-procedures.  As  a  result,  an  elemental  analysis  of 
crew  behavior  in  response  to  any  set  of  operational  tasks  should  begin 
with  a  lower  level  of  component  procedures. 

The  SOPe  listed  in  Table  6  begin  where  Predeparture  Planning  ends. 
They  display  various  levels  of  task  aggregation,  but  they  are  all  con¬ 
sidered  mandatory  for  normal  flight  operations  in  instrument  meteorolog¬ 
ical  conditions.  The  published  format  of  each  SOP  is  indicated  by  the 
letters  C  (checklist) ,  G  (graphical  or  pictorial)  ,  and  N  (narrative) . 

The  cockpit  crew  members  expected  to  exhibit  active  procedural  behavior 
are  identified  by  the  symbols  A  (all),  PI  (captain),  P2  (copilot),  FE 
(flight  engineer) ,  PF  (pilot  flying) ,  PNF  (pilot  not  flying) ,  and  U 
(unspecified).  As  a  general  rule,  whenever  operating  conditions  per¬ 
mit,  other  crew  members  are  supposed  to  passively  monitor  acceptable 
behavior  and  actively  intercede  to  proscribe  substandard  behavior. 
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TABLE  6 


NORMAL  OPERATING  PROCEDURES 


Index 

Number 

Name 

Format 

codes 

Operator 

codes 

1. 

Basic  ATC  communications  practices 

N 

PNF 

2. 

Preflight  Radio  Checklist 

C,N 

P2 

3. 

Gear  pin  status  report 

N 

PI 

4. 

Hydraulic  system  pressurization 

N 

PI  &  FE 

5. 

ATIS  report 

N 

U 

6. 

Clearance  Delivery  communications 

N 

PNF 

7. 

Ground  Control  communications 

N 

PNF 

8. 

Pre-start  Checklist 

C,N 

A 

9. 

Ground  crew  report 

N 

PI 

10. 

Cabin  report 

N 

PI 

11. 

Engineer's  Start  Checklist 

C,N 

FE 

12. 

Start  Checklist 

C,N 

A 

13. 

Engine  starting 

N 

PI  &  FE 

14. 

Ground  connections  and  hand  signals 

report 

N 

PI 

15. 

Engineer's  Taxi  Checklist 

C,N 

FE 

16. 

Pre-taxi  Checklist 

C,N 

A 

17. 

Transfer  of  EGT  monitor 

N 

PI  &  FE 

18. 

Ground  Control  communications 

N 

PNF 

19. 

Taxi 

N 

PF 

20. 

Takeoff  and  departure  briefing 

N 

PF 

21. 

Final  weight  and  balance  computation 

G,N 

FE 

22. 

Taxi  Checklist 

C,N 

A 

23. 

Tower  communications 

N 

PNF 

24. 

Passenger  pre-takeoff  announcement 

N 

PF 

25. 

Engineer's  Takeoff  Checklist 

C,G,N 

FE 

26. 

Runway  line  up 

N 

PF 

27. 

Takeoff  Checklist 

C,N 

A 

28. 

Thrust  setting  (takeoff  power) 

G,N 

PF  &  FE 

29. 

Takeof  f 

N 

PF 

30. 

Takeoff  callouts 

N 

PNF 

31. 

Noise  abatement  departure 

G,  N 

PF 

32. 

Gear  retr;  ction 

N 

PF  &  PNF 

33. 

Departure  Control  communications 

(initial  contact) 

N 

PNF 

34. 

Thrust  setting  (rated  power) 

G,N 

PF  &  FE 

35. 

Departure  Control  communication 

(radar  vector) 

N 

PNF 

36. 

Flap  retraction 

G,N 

PF  &  PNF 

37. 

Altitude  callout 

N 

PNF 

38. 

Intermediate  level  off 

N 

PF 

39. 

Departure  Control  communications 

(climb  clearance) 

N 

PNF 
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Table  6  (Contd.) 


Index 

Format 

Operator 

Number 

Name 

codes 

codes 

40. 

Airways  navigation  practices 

N 

PF  &  PNF 

41. 

Thrust  setting  (rated  power) 

G,N 

PF  &  FE 

42. 

Climb  (below  10,000  feet  MSL) 

N 

PF 

43. 

Company  departure  report 

N 

0 

44. 

ARTCC  communications  (initial  contact) 

N 

PNF 

45. 

Seat  belt  sign 

N 

U 

46. 

Climb  (above  10,000  feet  MSL) 

N 

PF 

47. 

After  Takeoff  Checklist 

C,N 

PF  &  FE 

48. 

Altimeter  reset  [not  applicable  for 

cruising  below  18,000  feet] 

N 

A 

49. 

ARTCC  communications  (route  clearance) 

N 

PNF 

50. 

Cruise  data 

G,N 

FE 

51. 

Altitude  callout 

N 

PNF 

52. 

Level  off 

N 

PF 

53. 

Mach  number/air speed  crosscheck 

N 

FE 

54. 

Cruise 

N 

PF 

55. 

ARTCC  communications  (radar  vector) 

N 

PNF 

56. 

Turbulence  penetration 

N 

A 

57. 

ARTCC  communications  (radar  vector) 

N 

PNF 

58. 

Turbulence  exit 

N 

A 

59. 

ARTCC  communications  (route  clearance) 

N 

PNF 

60. 

Fuel  systems  management 

G,N 

FE 

61. 

ARTCC  communications  (center  change; 

initial  contact) 

N 

PNF 

62. 

ATIS  report 

N 

U 

63. 

Company  arrival  report 

N 

U 

64. 

Approach  briefing 

N 

PF 

65. 

ARTCC  communications  (sector  change; 

initial  contact) 

N 

PNF 

66. 

Approach  data  and  speed  bugs 

G,N 

A 

67. 

Passenger  arrival  announcement 

N 

PF 

68. 

Descent  Checklist 

C,N 

A 

69. 

ARTCC  communications  (descent  clearance) 

N 

PNF 

70. 

Descent  (above  10,000  feet  MSL) 

N 

PF 

71. 

Altimeter  reset 

N 

A 

72. 

Seat  belt  sign  and  landing  lights 

N 

U 

73. 

Descent  (below  10,000  feet  MSL) 

N 

PF 

74. 

Approach  Control  communications 

(initial  contact;  clearance) 

N 

PNF 

75. 

Approach  Checklist 

C,N 

A 

76. 

Category  I  Instrument  Landing  System 

(ILS)  Approach 

G,N 

PF 

77. 

Approach  radio  checks 

N 

PF  &  PNF 

Table  6  (Contd.) 


Index 

Number 

Name 

Format 

codes 

Operator 

codes 

78. 

Altitude  callout 

N 

PNF 

79. 

No  smoking  sign 

N 

U 

80. 

Approach  Control  communications 
(radar  vector) 

N 

PNF 

81. 

Approach  flap  extension 

G,N 

PF  &  PNF 

82. 

Course  bar  and  glide  slope  callouts 

N 

PNF 

83. 

Approach  Control  communications 
(approach  clearance) 

N 

PNF 

84. 

Landing  gear/landing  flap  extension 

G,N 

PF  &  PNF 

85. 

Landing  Checklist 

C,N 

A 

86. 

Final  approach  fix  (FAF)  communications 

N 

PNF 

87. 

FAF  instrument  crosscheck 

N 

PNF 

88. 

Precision  approach  callout 

N 

PNF 

89. 

Outside  scan  and  visibility  callouts 

N 

PNF 

90. 

Landing 

N 

PF 

91. 

Landing  roll  callouts 

N 

PNF  &  FE 

92. 

Tower  communications 

N 

PNF 

93. 

After  Landing  Checklist 

C,N 

A 

94. 

Taxi 

N 

PF 

95. 

Ground  Control  communications 

N 

PNF 

96. 

Parking 

N 

PF 

97. 

Blocks  Checklist 

C,N 

A 

The  foregoing  list  does  not  include  "optional”  procedures  such  as 
those  pertaining  to  autopilot  tasks  or  the  transfer  of  aircraft  con¬ 
trol  between  pilots.  The  former  category  was  of  particular  interest 
to  Dr.  Ruffell  Smith;  the  latter  is  employed  in  the  current  investiga¬ 
tion.  Instrument  scanning  procedures  for  each  operator,  although 
different  for  flight  versus  ground  operations,  are  omitted  because  of 
their  continual  repetition  from  engine  start  through  parking.  Un¬ 
counted  operating  limitations,  restrictions,  and  regulations  apply  to 
individual  crew  members  and  aircraft  subsystems.  These,  too,  are 
omitted.  Thus,  the  enumerated  SOPs  represent  a  lengthy,  albeit 
simplified  and  technically  incomplete,  standard  for  cockpit  activities 
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lasting  approximately  75  minutes.  The  first  segment  of  the  experi¬ 
mental  scenario  was  intended  to  elicit  all  but  the  last  five  Normal 
Operating  Procedures. 

Empirical  Taxonomies 

From  the  mid  1960s  through  the  mid  1970s  R.  B.  Miller  (30,31), 

E.  A.  Fleishman  (13),  W.  H.  Teichner  (58),  and  others  devoted  consid¬ 
erable  effort  to  the  development  of  human  task  taxonomies.  In  his 
well  known  paper  on  the  rationale  for  different  constructs  of  these 
taxonomies  Miller  started  with  a  basic  premise:  "a  taxonomy  is  a  means 
of  classifying  objects  or  phenomena  in  such  a  way  that  useful  relation¬ 
ships  among  them  are  established."  (30:167)  His  statement  applies  as 
well  to  procedures  as  to  the  tasks  for  which  they  are  synthesized. 

The  Normal  Operating  Procedures  above  have  already  been  denoted 
as  belonging  to  an  empirical  or  colloquial  taxon  which  is  distinct 
from  the  Abnormal,  Alternate,  Irregular,  and  Emergency  classifications. 
The  status  of  aircraft  equipment  and  the  characteristics  of  the  oper¬ 
ating  environment  are  factors  which  usually  determine  which  class  of 
SOP  is  appropriate  in  a  given  situation.  There  is  extensive  grouping 
of  procedures  in  the  AOM  and  COM  according  to  these  taxa,  but  the 
segregation  is  not  perfect.  Moreover,  since  circumstances  may  require 
a  combination  of  these  procedural  types,  a  publication  devoid  of  cross 
references  is  probably  not  feasible.  While  the  colloquial  categories 
are  not  transcendent,  they  are  useful. 

Another  empirical  classification  scheme  is  determined  by  the 


agency  which  prescribes  or  enforces  an  SOP.  A  source  document 


taxonomy  is  a  further  refinement  of  the  same  idea.  The  purpose  served 
by  these  taxonomies  is  in  knowing  where  to  appeal  for  revisions  to 
unsatisfactory  procedures.  Not  incidentally,  operator  knowledge  of 
these  schemes  also  permits  compliance  effort  to  be  concentrated  on 
whichever  SOPs  are  in  a  particular  evaluator’s  jurisdiction  or  area  of 
interest.  The  three  agencies  responsible  for  the  vast  majority  of 
aircrew  procedures  are  the  FAA,  the  company  (airline) ,  and  the  aircraft 
manufacturer. 

Some  SOPs  clearly  belong  to  one  organization  or  the  other.  Speed 
restrictions  in  controlled  airspace  are  published  and  enforced  by  the 
FAA.  The  pre-arrival  radio  message  to  Operations  is  decreed  by  the 
company.  Many  more  SOPs,  for  example,  checklists,  are  hybrids.  They 
are  comprised  of  steps  partially  designed  by  the  aircraft  manufacturer 
with  the  approval  of  the  FAA  and  partly  included  at  the  company's 
initiative.  Irrespective  of  the  source  of  SOPs,  the  foremost  issue  in 
this  research,  as  in  many  accident  investigations,  is  the  type  and 
extent  of  aircrew  noncompliance. 

Two  other  organizations  have  the  ability  to  influence  the  design 
of  operational  procedures.  Either  group's  dominance  in  the  regulatory 
domain  could  produce  profound  changes  in  operator  behavior  and  flight 
safety.  They  are  the  pilots’  and  controllers'  unions,  ALPA  (the  Air 
Line  Pilots  Association)  and  PATCO  (the  Professional  Air  Traffic 
Controllers'  Organization) .  Since  none  of  the  SOPs  for  the  experi¬ 
mental  scenario  is  known  to  be  directly  attributable  to  either  agency, 
the  importance  of  their  input  to  SOPs  is  impossible  to  assess. 
Recognition  of  potential  taxa  attributable  to  them  merely  implies 


additional  layers  of  complexity,  in  accounting  for  aircrew  or  con¬ 
troller  behavior. 
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A  third  empirical  taxonomy  is  one  based  on  arbitrary  definitions 
of  the  operational  phases  of  a  mission  segment.  The  taxa  are  varia¬ 
tions  on  the  names  of  the  aggregated  control  procedures  previously 
mentioned.  The  Normal  Operating  Procedures  are  easily  reclassified 
according  to  this  taxonomy.  The  class  names  and  membership  are  shown 
in  Table  7. 


TABLE  7 

MISSION  PHASE  TAXONOMY 


Mission  Phase 


Normal  Operating  Procedures  (by  index  number) 


Pre-start 

Start/Pre-taxi 

Taxi  for  takeoff 

Takeof  f / Departure 

Clumb 

Cruise 

Descent 

Approach 

Landing 

Taxi  after  Landing 
Parking/Shutdown 


1  through  12  (total  12) 

1,  13  through  18  (total  7) 

1,  19  through  23  (total  6) 

1,  24  through  39  (total  17) 

1,  40  through  51  (total  13) 

1,  40,  52  through  59  (total  20) 
1,  40,  70  through  74  (total  7) 
1,  75  through  83  (total  10) 

1,  84  through  91  (total  9) 

1,  92  through  95  total  5) 

96  and  97  (total  2) 


Miller  has  observed  that  the  usefulness  of  a  taxonomy  is  not  pre¬ 
dicated  upon  rigor,  in  other  words  mutually  exclusive  taxa.  (30:168) 
The  mission  phase  taxonomy  illustrates  that  point.  Several  SOPs 
appear  in  more  than  one  phase.  The  purpose  is  to  give  a  coarse  indi¬ 
cation  of  the  distribution  of  Normal  Procedures  throughout  a  fairly 
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typical  flight,  namely  the  first  segment  of  simulated  mission.  It  is 
not  intended  to  be  a  measure  of  procedural  load.  SOPs  vary  too  much 
in  content,  duration,  and  criteria  to  let  their  total  numbers  be  con¬ 
sidered  as  weights.  The  mission  phase  taxa  point  to  the  need  for  more 
analytical  classifications. 

Analytical  Taxonomies 

The  ground  work  for  two  analytical  taxonomies  has  already  been 
laid.  The  earlier  annotations  concerning  presentation  format  and  crew 
member  involvement  lead  directly  to  informative  taxa. 

a.  Format 

Checklists,  graphs  or  pictorials,  narratives,  and  combinations 
thereof  are  the  explicit  formats  for  aircrew  SOPs.  Checklists  and 
graphs  never  occur  in  isolation;  they  are  always  accompanied  by 
explanatory  material.  In  contrast,  narrative  SOPs  frequently  stand 
alone.  Crew  members  have  no  quick  reference  memory  aids  for  narrative 
procedures.  Presumably,  they  do  not  need  any. 

Checklists,  on  the  other  hand,  are  tangible  reminders.  Fifteen 
of  them  are  applicable  to  a  normal  flight  segment.  They  are  scattered 
among  nearly  all  the  operational  phases  of  the  flight.  At  least  one 
checklist  appears  in  ten  of  the  eleven  previously  identified  mission 
phases.  Six  phases.  Pre-start,  Pre-taxi,  Taxi  for  Takeoff,  Takeoff, 
Approach,  and  Landing,  are  in  many  respects  anchored  around  checklists 
with  similar  names. 

Only  five  checklists  require  less  than  complete  crew  participa¬ 
tion.  Three  of  those  are  restricted  to  flight  engineer  initiative  and 


activity.  A  fourth  (Preflight  Radio)  is  designed  for  accomplishment 
by  the  copilot.  The  fifth,  the  After  Takeoff  Checklist,  specifies 
flight  engineer  activities  at  the  behest  of  the  flying  pilot.  The 
other  ten  checklists  all  prescribe  complete  crew  participation.  The 
implicit  requirement  for  coordination  among  unique  personalities  and 
roles  brings  these  SOPs  to  the  forefront  of  subsequent  behavioral 
analysis. 

Thirteen  normal  cockpit  procedures  are  at  least  partially  in 
pictorial  format.  Portions  of  the  Noise  Abatement  Departure,  Flap 
Retraction,  ILS  Approach,  Approach  Flap  Extension,  and  Landing  Gear / 
Landing  Flap  Extension  procedures  utilize  diagrams  or  plan  view  draw¬ 
ings  to  convey  procedural  information  to  crew  members.  A  generalized 
ILS  approach  depiction  appears  as  Figure  2  of  the  Appendix  C.  The 
The  specific  ILS  approach  plate  for  JFK  Runway  4R  is  also  shown  in  the 
Appendix  C  (Figure  3) .  Although  a  picture  may  not  literally  be 
"worth  a  thousand  words,"  graphical  and  tabular  formats  are  unques¬ 
tionably  efficient  ways  of  packaging  procedural  imperatives.  The  pilots 
are  responsible  for  executing  these  SOPs  and  for  integrating  them  with 
others  as  required. 

The  flight  engineer  has  the  primary  responsibility  for  compliance 
with  the  procedures  concerning  weight,  center  of  gravity,  thrust,  and 
fuel  consumption  computations.  Each  of  these  SOPs  is  in  some  manner 
dependent  on  interpretation  of  graphical  or  tabular  data.  A  simple 
chart  displaying  expected  aircraft  performance  parameters  for  a  .84 
Mach  cruise  condition  is  shown  in  Table  25  of  Appendix  C.  All  oper¬ 
ators  must  possess  visual  acuity  as  well  as  cognitive  and  motor 
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agility  to  comply  with  procedures  in  pictorial  format. 

All  of  the  Normal  Procedures,  including  checklists,  have  narrative 
elements.  The  same  is  true  of  Alternate  and  Emergency  Procedures; 
however,  a  number  of  Abnormal  Procedures,  including  these  concerned 
with  hydraulic  system  failures,  fuel  jettisoning,  and  inflight  engine 
shutdown/restart,  are  presented  exclusively  in  checklist  format. 

When  viewed  in  terms  of  the  major  activities  prescribed,  the 
Normal  Procedures  can  be  viewed  as  three  large  families,  one  small 
family,  and  a  few  unattached  offsprings.  The  first  family  contains 
aircraft  control  procedures,  both  highly  aggregated  and  otherwise.  In 
general,  they  are  of  relatively  long  duration,  they  necessitate  few 
communications,  and  they  can  seemingly  be  executed  in  parallel  with 
other  SOPs.  A  second  family  consists  of  extra-cockpit  audio  communi¬ 
cations.  They  are  of  short  duration,  often  just  a  matter  of  seconds; 
and  they  usually  require  the  absence,  interruption,  or  termination  of 
other  activity  by  one  operator  to  complete.  Intra-cockpit  coordination 
procedures  are  in  the  third  family.  Briefings,  checklists,  performance 
parameter  callouts,  control  transfer,  and  some  configuration  changes 
are  members  of  this  family.  The  durations  of  procedural  behavior  have 
high  variability,  and  their  characteristics  regarding  parallel  activity 
are  not  consistent. 

The  smallest  family  consists  of  six  procedures  which  are  related 
to  aircraft  subsystems  and  are  reserved  for  the  flight  engineer.  There 
are  three  silent  checklists  (index  numbers  11,  15,  and  25)  and  three 
silent  numerical  computations  (index  numbers  21,  53,  and  60). 
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The  remaining  SOPs  lack  strong  ties  to  any  one  family.  The 
passenger  seat  belt  and  no  smoking  sign  procedures  deal  more  with 


general  safety  than  with  aircraft  control,  communications,  or  subsystem 
operation.  In  contrast,  the  altimeter  resetting  procedures  possess 
characteristics  of  each  of  the  four  other  families.  They  simultane¬ 
ously  belong  to  every  family  and  to  none. 

In  summary,  a  taxonomy  based  on  procedural  format  provides  an 
appreciation  of  the  mental  flexibility  and  integrative  capacity  which 
a  crew  of  "by  the  book"  aviators  must  exercise.  The  classifications 
described  above  are  condensed  into  Tables  8,  9  and  10. 


TABLE  8 


FORMAT  TAXONOMY 


Format 


Normal  Operating  Procedures  (by  index  number) 


Check  list  only 


Narrative  only 

1, 

3-7,  9, 

10, 

13, 

,  14,  17 

-20, 

23, 

24, 

26, 

29, 

30 

32, 

,  33,  35 

»  37 

-40, 

,  42-46, 

48, 

49, 

51, 

52, 

54- 

59, 

61- 

-65,  67, 

69- 

74, 

77-80, 

82, 

83, 

86-92 

,  94 

-96 

Checklist  and 

2, 

8,  11, 

12, 

15, 

16,  22, 

27, 

47, 

ON 

00 

75, 

85, 

93 

narrative  97 


Graph  and 
narrative 


21,  28,  31,  34,  36,  41,  50,  53,  60,  66,  76,  81, 
84 


Checklist,  graph,  25 
and  narrative 


There  are  no  "checklist  only"  Normal  Procedures,  but  that  format 
does  appear  among  the  Abnormal  Procedures. 


TABLE  9 


NORMAL  CHECKLIST  PROCEDURES  CLASSIFIED  BY  MISSION  PHASE 

Mission  Phase  Checklist  Procedures  (by  index  number) 

Pre-start  2,  8,  11,  12 


Start/Pre-taxi 

15,  16 

Taxi  for  Takeoff 

22 

Takeoff /Departure 

25,  27 

Climb 

47 

Cruise 

68 

Descent 

— 

Approach 

75 

Landing 

85 

Taxi  after  Landing 

93 

Parking /Shutdown 

97 

TABLE  10 

FAMILIES  OF  NARRATIVE  PROCEDURES 


Family  Narrative  Procedures  (by  index  number) 


Aircraft  control 

19, 

26, 

29, 

31, 

38, 

40, 

42 

,  46 

,  52 

,  54, 

56 

w 

Ln 

00 

70, 

73, 

76, 

90, 

94, 

96 

Extra-cockpit 

1, 

3-7, 

9, 

10, 

14, 

18, 

23, 

24, 

33, 

35, 

39, 

43, 

communications 

44, 

49, 

55, 

57, 

59, 

61, 

62 

,  63 

,  65 

,  67, 

69 

,  74, 

80, 

83, 

86, 

92, 

95 

Intra-cockpit 

2, 

8,  12 

:,  13,  16,  17,  20, 

22, 

27, 

28,  30, 

32, 

communications 

34, 

36, 

37, 

41, 

47, 

50, 

51 

,  64 

,  66 

,  68, 

75 

,  77, 

78, 

81, 

82, 

84, 

85, 

87, 

88 

,  89 

,  91 

,  93, 

97 

Aircraft  subsystems 

11, 

15, 

21, 

25, 

53, 

60 

Unattached 

45, 

00 

71, 

72, 

79 
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b.  Operator  Involvement 

Personal  supervision  of  military  transport  crews  and  analysis  of 
the  full  mission  flight  simulator  data  have  convinced  this  author  of 
the  importance  of  yet  another  taxonomy  of  procedures.  This  classifi¬ 
cation  is  determined  by  the  prescribed  operator  involvement  with 
individual  SOPs.  The  purpose  is  to  recognize  predominate  responsbil- 
ities  for  required  crew  activity,  with  emphasis  on  manual  control  and 
communications.  Pure  monitoring  activity  is  disregarded  in  this 
taxonomy.  Justification  for  the  omission  of  monitoring  behavior  is 
based  on  the  assumption  that  any  crew  member  not  executing  assigned 
motor  tasks  should  be  periodically  monitoring  the  performance  of  all 
system  components,  including  other  humans. 

The  distribution  of  Normal  Operating  Procedures  according  to 
operator  involvement  is  as  follows:  4  for  the  captain,  1  for  the 
copilot,  7  for  the  flight  engineer,  18  for  the  flying  pilot,  30  for 
the  nonflying  pilot,  3  for  the  captain  and  flight  engineer  together, 

6  for  the  two  pilots  together,  4  for  the  PF  and  FE  jointly,  1  for  the 
PNF  and  FE  jointly,  15  for  the  whole  crew,  and  7  for  unspecified 
operators.  By  themselves  these  numbers  do  not  provide  meaningful 
comparisons  of  operator  load. 

For  instance,  the  procedures  assigned  to  the  PF  and  PNF  are  just 
not  comparable.  Approximately  two-thirds  of  the  PF's  unassisted 
activities  concern  aircraft  control  while  a  similar  portion  of  the 
PNF's  tasks  have  to  do  with  external  communicat ions .  This  basically 
distinct  division  of  labor  is  somewhat  compromised  by  the  fact  that  a 
total  of  21  SOPs  dictate  challenge  and  response  coordination  between 
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the  pilots  and  another  13  mandate  a  one-way  transfer  of  information. 

The  flight  engineer's  procedures  are  unlike  those  of  either 
pilot.  When  all  subsystems  are  working  properly,  the  FE  is  clearly  a 
supporting  actor.  Although  he  cooperates  on  23  SOPs  with  one  or  both 
of  the  pilots,  he  has  only  seven  tasks  exclusively  reserved.  Under 
"normal"  operating  conditions  the  engineer  is  constrained  by  fewer 
procedures  than  are  the  pilots.  As  a  result,  he  may  appear  better 
able  to  comply  with  the  prescribed  SOPs.  Subjectively,  at  least,  the 
FE's  total  procedural  load  is  considerably  below  that  of  the  pilots. 
Table  11  summarizes  the  by-operator  taxonomy. 


TABLE  11 

BY-OPERATOR  TAXONOMY 


Operators 

Normal  Operating  Procedures  (by  index  number) 

Captain  (PI) 

3,  9,  10,  14 

Pilot  flying  (PF) 

19,  20,  24,  26,  29,  31,  38,  42,  46,  52,  54,  64, 
67,  70,  73,  76,  90,  94,  96 

Copilot  (P2) 

2 

Pilot  not  flying 
(PNF) 

1,  6,  7,  23,  30,  33,  35,  37,  39,  44,  49,  51,  57, 
59,  61,  65,  69,  74,  78,  80,  82,  83,  86,  87,  88, 
89,  92,  95 

Flight  Engineer  (FE) 

11,  15,  21,  25,  50,  53,  60 

PI  and  FE 

4,  13,  17 

PF  and  PNF 

32,  36,  40,  77,  81,  84 

PF  and  FE 

28,  34,  41,  47 

PNF  and  FE 

91 

PF,  PNF,  and  FE 

8,  12,  16,  22,  27,  48,  56,  58,  66,  68,  71,  75, 
85,  93,  97 

Unspecified 

5,  43,  45,  62,  63,  72,  79 
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Consideration  of  the  Normal  Operating  Procedures  in  light  of  the 
by-operator  taxa  discloses  the  dominant  responsibilities  of  the  pilot  • 
team  as  well  as  the  criticality  of  crew  coordination.  These  realities 
weigh  heavily  on  the  choice  of  which  behavioral  imperatives  to 
analyze.  One  other  fact  is  also  divulged;  namely,  despite  essentially 
equivalent  instruments  and  controls  at  the  two  pilot  stations,  the 
behavioral  roles  of  the  occupants  are  intended  to  be  vastly  different. 
One  pilot,  the  PF,  is  primarily  a  system  controller  and  monitor;  the 
other  (PNF)  is  a  facilitator,  compensator,  conciliator,  and  communi¬ 
cator.  For  normal  operations  the  procedural  distinctions  between  a 
captain  PNF  and  a  copilot  PNF  are  theoretically  nil.  Yet,  in  practice, 
the  distinctions  are  remarkable. 

The  expected  relationships  between  a  captain  and  the  rest  of  his 
crew  are  set  forth  in  two  explicit  narrative  procedures  (not  listed 
above) .  The  COM  policy  on  Command  and  the  separate  policy  on  Manage¬ 
ment  detail  basic  role  differences  within  the  crew.  The  authority  to 
command,  i.e.,  to  issue  instructions  and  require  compliance  is 
expressly  vested  in  the  pilot-in-coramand.  In  commercial  air  carrier 
operations  the  pilot-in-command  is  the  captain  whether  he  is  the 
flying  pilot  or  not.  Thus,  the  captain  can,  indeed  he  must,  "exercise 
full  control"  over  every  operational  activity  even  though  he  may  not 
be  manually  controlling  the  aircraft  or  its  subsystems.  The  potential 
for  role  conflict  in  a  captain  as  PNF  and  copilot  as  PF  situation  is 
omnious.  Lack  of  confidence,  cooperation,  or  empathy  between  the 
pilots  could  produce  nonstandard  behavior  by  one  crew  member  that  no 
amount  of  procedural  compliance  by  others  could  neutralize. 


In  further  elaborating  upon  operator  roles,  the  COM  offers  a 
flight  management  policy  apparently  designed  to  pacify  government 
regulators  and  subordinate  crew  members  without  substantively  lessen¬ 
ing  the  captain's  authority.  The  policy  calls  for  management  accord¬ 
ing  to  the  "crew  concept."  As  explained,  that  concept  necessitates 
full  communication  and  coordination  among  crew  members.  It  demands 
"constant  vigilance,  cross-checking,  and  sharing  of  information." 
However,  after  informing  the  captain  to  whatever  extent  is  possible, 
other  crew  members  are  required  to  give  full  support  to  his  directions 
and  decisions.  (It  should  be  pointed  out  that  company  procedures  are 
typically  drafted  and  approved  by  senior  captains  in  executive 
positions.) 

The  problem  of  striking  a  proper  balance  among  authority,  explicit 
SOPs,  and  subordinate  crew  member  responsibilities  faces  all  opera¬ 
tional  managers.  The  COM  summarizes  one  airline's  approach  to  the 
problem  in  a  single  paragraph. 

The  use  of  standard  procedures  and  terminology  promotes 
confidence  and  precision  within  the  crew.  The  level  of 
standardization  must  be  high  enough  to  discourage  un¬ 
safe  practices  and  carelessness  but  should  not  limit 
operational  flexibility  unnecessarily  or  discourage  the 
use  of  good  judgment. 

The  Ruff ell  Smith  data  show  how  this  company  philosophy  is  actually 
applied  in  the  operational  environment. 

Crew  Coordination  Procedures 

Since  a  captain's  position  of  leadership  is  well  documented  and 
since  crew  coordination  is  in  many  respects  dependent  upon  exemplary 
leadership,  a  crew's  procedural  behavior  or  the  lack  thereof  can  be 
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a  guage  of  the  strength  of  internal  leadership.  A  subset  of  the  Normal 
Operating  Procedures  which  elicit  verbal  interactions  between  a  cap¬ 
tain  and  his  crew  has  been  chosen  for  further  examination.  A  complete 
listing  is  shown  in  Table  12. 


TABLE  12 

CREW  COORDINATION  PROCEDURES 
(To  be  used  for  quantitative  compliance  assessments) 


Index 

Letter 

A. 

B. 

C. 

D. 

E. 

F. 

G. 

H. 

I. 

J. 

K. 

L. 

M. 

N. 

O. 

P. 

Q. 

R. 

S. 

T. 


Procedure  Name 

Pre-start  Checklist 

Start  Checklist 

Pre-taxi  Checklist 

Transfer  of  EGT  Monitor 

Taxi  Checklist 

Takeoff  Checklist 

Takeoff  Callouts 

Gear  Retraction 

Flap  Retraction 

Altitude  Callout 

After  Takeoff  Checklist 

Altitude  Callout 

Transfer  of  Aircraft  Control 

Descent  Checklist 

Approach  Checklist 

Altitude  Callout 

Approach  Flap  Extension 

Landing  Gear/Landing  Flap  Extension 

Landing  Checklist 

Precision  Approach  Callouts 


Landing  Roll  Callouts 


U. 
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These  crew  coordination  SOPs  are  published  in  all  of  the  vari¬ 
ous  formats  described  above.  They  occur  in  eight  of  the  eleven 
mission  phases.  Descent  is  not  represented  because  of  its  short 
duration  in  the  scenario.  No  data  were  collected  for  the  Taxi  after 
Landing  or  Parking/Shutdown  phases.  In  most  cases,  the  two  pilots  are 
the  main  actors,  but  flight  engineer  participation  is  not  ignored. 

The  optional  Trans^  -  of  Aircraft  Control  procedure  is  included 
because  its  use  implies  the  flying  pilot's  willingness  to  trust  in  and 
to  share  responsibility  with  the  other  pilot. 

Eight  of  the  nine  checklists  require  full  crew  participation. 

Each  of  the  eight  is  to  be  accomplished  via  challenge  and  response 
interactions.  The  copilot  and  the  flight  engineer  are  the  usual 
challengers.  Of  particular  interest  are  the  identity  of  the  crew 
member  who  initiates  each  checklist,  the  manner  of  initiation, and  the 
meticulousness  of  both  challengers  and  respondents.  According  to  the 
COM  the  captain  or  the  PF  should  initiate  normal  operating  checklists, 
and  "in  most  instances"  prescribed  activity  should  be  accomplished 
before  the  challenges  begin.  When  used  in  this  manner,  a  checklist 
helps  to  notify  all  crew  members  of  task  status.  A  different  but 
familiar  use  of  a  checklist  is  as  a  triger  for  commencement  of  task 
activity.  The  COM  does  not  condone  such  use,  and  it  specifically 
prohibits  continuation  of  checklist  challenges  beyond  any  incomplete 
sub-task. 

Normal  operating  checklists  are  to  be  initiated  and  completed 
within  established,  control-oriented  time  frames.  They  are  supposed 

i 

to  be  started  so  that  adequate  time  is  available  to  accomplish  all  ] 
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activities  without  interruption.  This  is  not  possible  in  either  the 
simulated  or  the  real  world  operating  environments.  Radio  communica¬ 
tions  and  manual  control  requirements  force  interruptions.  Regardless, 
challenge  and  response  checklists  are  to  be  completed  without  devi¬ 
ation,  including  a  statement  of  the  checklist's  name  prior  to  the 
first  challenge  and  an  announcement  of  procedure  completion  following 
the  last  challenge. 

The  procedures  (index  letters  G,  J,  L,  P,  T,  and  U  in  Table  12) 
mandating  callout  of  aircraft  performance  parameters  at  specified 
values  do  not  require  acknowledgement  by  any  other  crew  member.  The 
nonflying  pilot  is  responsible  for  all  callouts  except  two.  Those  two 
occur  during  the  landing  roll.  They  concern  the  status  of  thrust 
reversers,  and  they  are  to  be  made  by  the  flight  engineer.  Compliance 
with  callout  procedures  demands  perceptual  diligence.  It  also  indi¬ 
cates  a  commitment  to  the  basic  flight  management  policy  stated  in  the 
COM. 

Of  the  six  remaining  procedures  four  embody  changes  in  the 
configuration  of  the  aircraft  (gear  or  flap  extension  and  retraction). 
The  importance  of  pilot  coordination  and  agreement  concerning  such 
changes  is  obvious.  Execution  of  the  other  two  procedures.  Transfer 
of  EGT  Monitor  and  Transfer  of  Aircraft  Control,  conveys  primary 
accountability  for  essential  monitoring  and  control  functions  from 
one  operator  to  another.  Again,  the  need  for  complementary  behavior 
is  apparent.  In  all  six  procedures  verbal  communication  by  two  crew 
members  is  mandatory.  The  Transfer  of  EGT  Monitor  responsibility  is 
between  the  pilot  who  starts  the  engines  (normally  the  captain)  and 
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the  flight  engineer,  usually  following  the  latter's  pre-taxi  checks. 
Transfer  of  Aircraft  Control  is  supposed  to  occur  whenever  the  flying 
pilot  must  divert  his  attention  to  another  task.  The  other  task  might 
employ  a  prescribed  SOP  such  as  briefing  the  crew,  or  an  implicit, 
personalized  procedure  such  as  studying  an  avigation  chart. 

The  interaction  or  crew  coordination  procedures  identified  here 
capture  essential  ingredients  of  group  leadership,  crew  management,  and 
behavioral  conformity.  Objective  data  pertaining  to  crew  compliance 
with  these  SOPs  have  not  previously  been  compiled.  Although  Dr,  Ruffell 
Smith  recognized  problems  of  crew  integration,  he  concentrated  on  errors 
in  control  of  aircraft  parameters  and  operation  of  subsystems.  He  did 
not  elaborate  on  the  possibility  of  a  relationship  between  meticulous 
compliance  with  coordination  procedures  and  his  determinations  of 
crew  error.  The  next  chapter  will  explore  that  possibility. 


CHAPTER  IV 


COMPLIANCE  ASSESSMENTS 

This  chapter  looks  at  compliance  with  twenty-one  crew  coordina¬ 
tion  procedures  from  two  very  biased  vantage  points.  The  first 
perspective  is  that  of  a  person  who  believes  qualified  crew  members 
should  have  the  benefit  of  any  doubt  concerning  the  suitability  of 
procedures.  From  that  point  of  view  explanations  of  noncompliant 
aircrew  behavior  are  sought  in  the  procedures  themselves  or  in  the 
circumstances  encompassing  their  use.  The  alternative  perspective  is 
that  of  an  evaluator  who  thinks  that  existing  procedures  are  basically 
flawless.  With  that  belief  the  assessment  of  individual  and  group 
performance  hinges  on  operator  compliance  with  a  collection  of  SOPs. 
Aberrant  behavior  is  ascribed  to  particular  operators,  and  patterns 
of  nonstandard  behavior  may  be  symptoms  of  aptitude  or  training 
deficiencies . 

In  the  latter  part  of  this  chapter  the  crew  errors  previously 
identified  by  Ruffell  Smith  are  reconfirmed.  They  are  also  recategor¬ 
ized  according  to  operator  involvement,  like  the  procedures  in 
Chapter  III.  New  error  statistics  are  computed  and  their  distributions 
are  studied.  Indicators  developed  to  describe  individual  and  crew 
compliance  with  coordination  procedures  serve  as  predictors  of  cate¬ 
gorical  error  counts.  The  implications  of  several  statistical  rela¬ 
tionships  are  discussed. 
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Procedures  and  Circumstances 

The  crew  coordination  procedures  identified  in  Chapter  III  are 
the  basis  of  a  quantitative  assessment  of  compliance  across  crews. 

The  previously  noted  subdivisions  (checklists,  callouts,  configuration 
changes,  and  transfers)  form  natural  groupings  for  diagnosing  unsatis¬ 
factory  procedural  imperatives  and  locating  circumstantial  anomalies. 

a.  Checklists 

The  Pre-start,  Start,  Pre-taxi,  Taxi,  and  Takeoff  Checklists  are 
supposed  to  be  initiated  upon  command  of  the  captain  or  the  flying 
pilot.  The  other  pilot  is  to  then  announce  the  name  of  the  checklist, 
presumably  as  a  confirmation  of  the  command,  and  read  the  opening 
challenge.  Once  initiated,  checklists  may  be  delayed  by  interruptions 
from  outside  the  cockpit  or  from  within;  but  the  checklist  must 
ultimately  be  resumed  and  completed  in  toto.  In  every  experimental  run 
this  requirement  is  met.  The  requisite  challenges  are  made  and  a 
response  is  given  for  each  challenge.  However,  some  of  the  operator 
actions  and  replies  are  contrary  to  procedural  specifications.  Dr. 
Ruffell  Smith  notes  several  such  actions  as  errors,  but  he  does  not 
observe  the  verbal  requirements  of  proceduralized  crew  coordination. 

The  first  four  checklists  named  above  are  entirely  challenged 
by  one  of  the  pilots.  Challenges  in  the  Takeoff  Checklist  are  begun 
by  a  pilot  and  completed  by  the  flight  engineer.  After  a  proper 
response  to  the  final  challenge  in  each  procedure,  the  challenger  is 
directed  to  state  "[name]  Checklist  is  complete."  This  concluding 
statement  is  printed  at  the  bottom  of  each  checklist.  The  Descent, 
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Approach,  and  Landing  Checklists  are  similar  to  those  above  except  that 
the  flight  engineer  is  the  sole  challenger.  The  After  Takeoff  Check¬ 
list  is  unlike  the  rest  because  there  are  no  verbal  challenges;  the 
engineer  accomplishes  all  steps  silently  and  merely  verifies  completion 
to  the  pilots. 

Among  the  ten  sampled  crews  there  are  remarkable  differences  in 
the  patterns  of  behavior  associated  with  who  the  prescribed  challenger 
is.  Considering  the  first  five  checklists,  or  a  total  of  fifty  oppor¬ 
tunities  over  ten  flights,  the  command-announcement-challenge  sequence 
is  fully  executed  only  five  times.  Two  occurrences  involve  the  Pre¬ 
start  Checklist;  three  are  on  the  Pre-taxi  Checklist.  Table  13  con¬ 
tains  the  data  for  all  pilot-challenged  checklists.  The  assortment  of 
noncompliant  behaviors  includes:  23  occasions  when  the  SOP  is  initiated 
without  any  command,  22  occasions  when  there  is  a  command  but  no 
confirmation  announcement,  and  13  occasions  when  there  is  neither  a 
command  nor  an  announcement. 

These  behaviors  raise  two  questions  about  the  prescribed  pro¬ 
cedural  communications.  First,  should  an  explicit  command  be  required 
to  initiate  a  normal  operating  checklist?  The  data  clearly  reveal  that 
checklist  tasks  can  be  accomplished  without  a  specific  command  or 
announcement.  In  circumstances  necessitating  conservation  of  time, 
such  shortcuts  seem  to  be  widely  accepted  expedients.  But  when  time 
constraints  are  not  severe,  similar  omissions  appear  to  strain  crew 
cohesion.  One  shortcut  often  begets  another,  and  a  crew’s  framework  of 
mutual  trust  and  structured  leadership  may  undergo  progressive 
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deterioration.  Rapport  among  the  crew  of  experimental  run  number  six 
practically  vanishes  by  the  time  they  "land"  at  JFK. 

One  purpose  for  mandating  audible  checklist  initiations  lies  in 
the  sweeping  Command  and  Management  Procedures.  They  affirm  that 
responsibility  for  all  operational  activity  resides  in  the  captain. 
Even  if  he  expressly  delegates  a  portion  of  his  authority  to  another 
crew  member,  he  knows  that  he  cannot  dispense  the  final  responsibility 
for  operational  safety  to  anyone.  He  should  also  know,  though  the 
SOPs  do  not  state  it,  that  whenever  any  operator  exceeds  the  estab¬ 
lished  limits  of  authority  or  the  bounds  of  procedural  tolerance 
without  subsequent  negative  reinforcement,  that  behavior  is  more 
likely  to  be  repeated  and  emulated.  In  short,  checklist  commands  are 
necessary.  They  reduce  uncertainty  and  preserve  internal  order. 

The  second  question  about  checklist  communications  has  to  do  with 
the  challenger's  first  intonation.  Should  he  be  required  to  announce 
each  checklist  by  name?  Nearly  half  of  the  observed  pilot  challengers 
fail  to  comply.  Nevertheless,  the  existing  requirement  can  be  easily 
rationalized.  The  announcement  should  occur  regardless  of  the 
presence  or  absence  of  a  formal  initiatory  command.  If  a  command  is 
given,  then  the  announcement  serves  as  an  acknowledgement  to  the 
issuer  and  as  a  secondary  notification  to  the  flight  engineer.  If  no 
command  is  given,  then  the  announcement  becomes  an  internal  cue  to 
crew  members  engrossed  in  other  activities. 

In  some  cases  internal  cues  may  seem  superflous  .  For  example, 
in  the  experimental  scenario  the  simulator  controller  consistently 
hurries  the  crews  through  the  taxi  phase  and  into  takeoff  position. 
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Most  crew  members  appear  to  sense  the  need  for  rapid  coordinated  action. 
Consequently,  crew  meticulousness  in  adhering  to  the  formal  verbal 
preliminaries  of  the  Take-off  Checklist  is  exceedingly  lax.  Six  crews 
start  the  Takeoff  Checklist  without  command  or  announcement;  yet  no 
extraordinary  problems  are  encountered.  When  all  crew  members  share 
a  common  expectation  of  events,  internal  coordination  can  appear 
automatic. 

In  rather  stark  contrast  to  the  paucity  of  audible  checklist 
introductions  between  the  two  pilots  are  the  more  extensive  verbaliza¬ 
tions  when  the  flight  engineer  is  the  challenger.  Table  14  displays 
these  data.  Exactly  half  of  the  30  checklist  sequences  (Descent, 
Approach,  and  Landing)  represented  in  the  data  begin  in  the  prescribed 
command-announcement-challenge  order.  Only  one  of  the  30  is  missing 
the  initial  command.  Pilots  seem  to  recognize  the  importance  of  clear 
and  concise  instructions  to  a  teammate  who  works  outside  their  normal 
field  of  vision.  For  their  part,  the  flight  engineers  collectively 
make  as  many  announcements  (15)  in  their  30  opportunities  as  the 
pilots  do  in  50.  In  addition,  more  engineers  are  self-consistent. 

Three  of  them  omit  all  announcements,  and  three  others  omit  none.  The 
origins  or  stimuli  of  such  behavior  are  not  apparent,  but  the  absence 
of  true  standardization  is. 

One  other  aspect  of  checklist  behavior  deserves  special  mention. 

It  reinforces  the  previously  noted  differences  between  pilots 
and  engineers  as  challengers.  In  eight  of  the  forty  cases  when  a 
pilot  gives  the  last  challenge,  he  fails  to  follow  the  response  with 
the  prescribed  procedure  completion  statement.  This  is  considerably 
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different  from  the  two  omissions  in  fifty  opportunities  demonstrated  by 
the  flight  engineers.  Tables  15  and  16  display  these  data. 

There  are  at  least  four  plausible  explanations  for  the  observed 
disparities  in  operator  behavior.  First,  the  engineers,  or  perhaps 
the  whole  crew,  may  be  more  attentive  to  procedural  detail  in  flight 
(e.g.,  the  checklists  read  by  the  engineer)  than  they  are  on  the 
ground  (e.g.,  checklists  read  by  a  pilot).  Data  concerning  use  of  the 
After  Landing  and  Blocks  Checklist  might  have  shed  some  light  on  this 
notion.  Secondly,  the  flight  engineers,  by  training  or  nature,  may  be 
more  observant  of  detail  than  are  the  pilots.  In  this  author’s  own 
experience,  professional  flight  engineers  seem  somewhat  less  inclined 
to  "cut  corners"  than  do  some  of  their  more  aggressive  pilot  counter¬ 
parts.  Thirdly,  flight  engineers  may  simply  have  more  time  to  attend 
to  the  finer  points  of  procedures  than  do  pilots.  Generally  speaking, 
during  normal  operations  the  engineer  has  the  most  freedom  to  set  his 
own  pace.  Finally,  since  the  flight  engineer  is  removed  from  the 
pilots'  normal  fields  of  vision,  both  he  and  they  may  be  predisposed 
to  rely  on  formal  verbal  communications  for  initiation  and  termination 
of  checklists.  If  this  supposition  is  true,  crew  coordination  might 
be  improved  by  making  the  flight  engineer  the  challenger  of  all 
checklists. 

b.  Callouts 

Callouts  of  aircraft  parameters  by  the  nonflying  pilot  or  the 
engineer  to  the  rest  of  the  crew  represent  procedures  fundamentally 
different  from  checklists.  There  are  no  preparatory  commands  or 
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announcements;  there  is  no  statement  of  completion;  and  no  responses 
are  required.  In  all  but  one  case  the  nonflying  pilot  acts  as  a  back¬ 
up  or  second-level  visual  monitor  who  audibly  relays  operating  informa¬ 
tion  to  the  flying  pilot.  Airline,  FAA,  and  NTSB  advocacy  of  callout 
procedures  has  resulted  from  a  large  number  of  mishaps  in  which  the 
flying  pilot  seemed  to  be  distracted  or  overloaded  during  periods  of 
rapid  change  in  aircraft  parameters,  namely,  the  takoff,  climb, 
approach,  and  landing  phases. 

Each  of  the  flight  phases  just  mentioned  is  represented  by  a 
callout  procedure  in  the  data.  The  Takeoff  Callouts  include:  first 
instrument  indication  of  airspeed,  four  predetermined  speeds  during 
takeoff  roll,  instrument  indication  of  a  positive  rate  of  climb,  and 
an  altitude  of  800  feet  above  field  elevation  (APE) .  The  Altitude 
Callouts  are  to  occur  during  climb  and  descent  when  the  aircraft 
reaches  1000  feet  from  its  clearance  altitude.  (A  mechanical  Altitude 
Alert  System  is  also  installed  to  provide  aural  and  visual  notification 
when  the  aircraft  comes  within  900  feet  of  a  selected  altitude.) 

Four  reports  of  decreasing  altitude  and  confirmation  of  passing  the 
final  approach  fix  (FAF)  constitute  the  Approach  Callouts.  The  five 
Landing  Callouts  relate  to  thrust  reverser  position  (two  calls  by  the 
flight  engineer)  and  deceleration  (three  calls  by  the  nonflying  pilot) . 

As  a  group,  the  sampled  crews  are  more  uniform  in  their  compli¬ 
ance  with  callout  procedures  than  with  checklists.  Table  17  contains 
the  pertinent  data.  During  takeoff  only  one  crew  (run  number  six) 
fails  to  make  all  of  the  required  calls.  In  run  number  three  the 
flying  pilot  twice  preempts  the  nonflying  pilot.  The  remaining  eight 
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crews  behave  as  expected.  No  deficiencies  in  the  procedure  itself  are 
evident . 

Compliance  with  the  Approach  Callouts  is  similar.  Four  isolated 
omissions  are  detectable,  but  they  do  not  point  to  any  particular 
deficiency  in  the  procedure.  Data  for  the  first  three  callouts  during 
landing  appear  consistent  and  unaffected  by  the  scenario  (two  omissions 
by  one  captain  form  part  of  a  larger  pattern  of  noncompliance) .  How¬ 
ever,  the  last  two  callouts  in  that  procedure  are  sometimes  omitted 
because  of  the  intervention  of  the  simulator  controller  in  terminating 
the  first  segment. 

Only  the  Altitude  Callouts  during  climb  and  descent  are  treated 
with  the  type  of  casualness  seen  earlier  in  some  checklists.  Sixteen 
out  of  30  required  altitude  calls  either  are  not  made  or  are  made  at 
other  than  the  prescribed  time.  One  possible  reason  is  that  crew  mem¬ 
bers  perceive  negligible  safety  benefit  from  a  "high  altitude"  callout. 
In  isolated  situations  other  tasks  interfere  with  callouts,  but  in  gen¬ 
eral  the  mental  load  on  the  nonflying  pilots  is  not  especially  heavy  at 
these  points. 

Another  explanation  of  such  behavior  is  that  the  built-in  redund¬ 
ancy  of  the  mechanical  alerting  device  tends  to  make  operators  less 
diligent.  If  this  is  true  then  the  allocation  of  tasks  between  men 
and  machines  should  be  reconsidered  in  light  of  operational  experience 
as  well  as  theoretical  concepts.  The  "1000  feet  to  level  off"  callout 
requirements  may  be  excessive  and  demeaning.  Procedure  designers 
must  consider  the  psychological  as  much  as  the  physiological  effects 
of  their  prescriptions  to  human  operators.  Simple  modification  of  the 


73 


callout  altitude  such  as  "500  feet  to  level  off,"  might  have  several 
advantages.  The  mechanical  warning  at  900  feet  could  be  more  valued 
than  it  is  now.  The  nonflying  pilot's  callout  would  then  occur  without 


mechanical  back-up,  and  the  flying  pilot  would  have  an  additional 
short-suspense  prompt  for  control  action.  In  any  event,  extensive 
operational  testing  of  revised  altitude  callout  procedures  should  be 
completed  as  soon  as  possible. 

c.  Configuration  Changes 

The  rudimentary  procedures  for  gear  and  flap  extension/ retraction 
probably  do  not  require  validation.  Few  pilots  argue  with  the  desir¬ 
ability  of  verbally  commanding  and  acknowledging  impending  changes  in 
aircraft  configuration.  In  fact,  since  the  landing  gear  level  in  the 
Boeing  747  is  located  on  the  copilot's  side  of  the  instrument  panel, 
flying  captains  are  physically  dependent  on  copilot  assistance  in 
changing  the  position  of  the  undercarriage.  In  addition,  captains  and 
copilots  alike  rely  on  their  nonflying  counterpart  to  audibly  confirm 
and  manually  select  commanded  changes  in  flap  position. 

The  experimental  data  (Table  18)  reveal  no  deviations  from  estab¬ 
lished  oral  procedures  for  Gear  Retraction  and  a  single  instance  of 
noncompliance  for  Flap  Retraction.  Likewise,  the  Approach  Flap  Exten¬ 
sion  and  Landing  Gear/Landing  Flap  Extension  SOPs  exhibit  high  rates 
of  behavioral  conformity  (95%  and  90%  respectively).  A  total  of  four 
configuration  changes  (of  104  actually  made)  evidence  omission  of  one 
of  the  two  prescribed  verbalizations.  One  change  (from  flaps  1  to 
flaps  up)  is  made  without  comment  by  either  pilot.  None  of  these 
facts  provide  much  enlightenment  about  the  underlying  procedural 
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imperatives.  However,  the  integration  of  configuration  change  SOPs 
within  the  corresponding  flight  control  procedures  is  noteworthy. 

Aircraft  altitude  and  location  over  the  ground  vary  considerably 
at  the  initiation  point  of  selected  conf iguration  change  procedures. 

The  start  of  Flap  Retraction  occurs  as  early  as  the  "800  feet"  callout 
on  takeoff  and  as  late  as  the  intermediate  level  off  at  4000  feet 
during  departure.  Neither  of  these  extremes  conforms  to  the  Noise 
Abatement  Departure  procedure  which  is  to  be  accomplished  concurrently. 
Flight  control  parameters  at  the  commencement  of  the  Landing  Gear 
Extension  procedure  show  comparable  variability.  Aircraft  altitude 
ranges  from  approximately  3500  feet  down  to  1500  feet;  distance  from 
the  airport  lies  between  eleven  and  five  nautical  miles. 

The  foregoing  observations  illustrate  the  difficulty  in  isolating 
and  interpreting  a  small,  yet  meaningful,  number  of  human  performance 
variables.  This  study  does  not  incorporate  flight  control  parameters 
per  se  in  the  assessment  of  procedural  compliance.  Instead,  it  focuses 
on  verbal  behavior.  This  author  does  not  eschew  other  indicators  of 
operator  performance  or  crew  coordination.  The  ones  chosen  bear 
directly  on  crew  integration.  Aircraft  parameters  relate  indirectly. 
Verbal  behavior  has  intuitive  as  well  as  substantive  appeal  in 
assessing  leadership  and  teamwork.  The  fact  that  audible  indicators 
yield  no  new  truths  about  the  synchronization  of  configuration  change 
activity  does  not  discount  their  value  in  studying  other  tasks. 

d.  Transfers 

Verbal  indicators  of  compliance  with  the  Transfer  of  Exhaust  Gas 
Temperature  (EGT)  Monitor  and  the  optional  Transfer  of  Aircraft  Control 
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Procedures  are  more  revealing.  Both  SOPs  typify  the  quality  of  com¬ 
munications  between  specific  pairs  of  crew  members.  The  EGT  procedure 
is  designed  to  elicit  flight  engineer  initiative  in  lessening  pilot 
load  and  increasing  his  own  for  the  remainder  of  a  flight.  The  control 
transfer  procedure  is  intended  for  use  as  many  times  during  flight  as 
the  pilot  team  deems  appropriate. 

In  the  experimental  data  (Table  19)  two  of  the  ten  flight  engineers 
fail  to  advise  the  flying  pilot  when  he  can  relinquish  responsibility 
for  monitoring  EGT.  As  a  result  of  engineer  noncompliance  with  the 
SOP,  the  pilot  team  theoretically  carries  a  larger  procedural  load 
throughout  the  mission.  On  one  crew  (run  number  twelve)  the  engineer 
actually  declines  separate  requests  by  each  pilot  to  assume  EGT  moni¬ 
toring  during  taxi.  In  that  case  the  pilots  obviously  desire  to  reduce 
their  load;  but  the  engineer  repeatedly  says  he  is  unable  to  relieve 
them.  During  run  number  thirteen  EGT  monitoring  responsibility  is 
never  mentioned.  Neither  the  pilots'  awareness  of  their  load  nor  the 
reason  for  the  engineer's  failure  to  initiate  the  procedure  becomes 
evident.  Nevertheless,  since  eight  of  the  ten  observed  crews  have  no 
difficulty  whatsoever  in  complying  with  the  EGT  monitoring  procedure, 
it  must  be  assumed  that  the  procedure  itself  is  adequate. 

The  Transfer  of  Aircraft  Control  procedure  also  seems  well  con¬ 
ceived.  Eighty  per  cent  of  the  pilot  teams  go  through  at  least  one 
full  "double  transfer."  That  type  of  exchange  is  desirable  whenever 
the  flying  pilot  must  temporarily  divert  his  attention  from  aircraft 
control.  He  should  direct  the  other  pilot  to  assume  control  ("You 
have  it")  and  require  him  to  acknowledge  ("I  have  it") .  When  the 
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original  flier  is  ready  to  resume  control,  the  sequence  may  be  reversed 
(e.g.,  "OK,  I  have  it"  followed  by  "You  have  it").  The  importance  of 
knowing  who  has  control  of  the  aircraft  is  unmistakable.  Similarly, 
there  is  no  doubt  about  the  merit  of  periodically  transferring  control 
to  review  maps  and  charts,  to  conduct  crew  and  passenger  briefings,  or 
to  just  relax. 

In  this  regard,  the  most  striking  data  from  the  experiment  are 
that  in  runs  six  and  twelve  the  pilot  teams  never  once  transfer  con¬ 
trol.  Even  though  the  flying  pilots,  both  of  whom  happen  to  be  co¬ 
pilots,  are  unable  to  accomplish  other  mandatory  tasks,  no  member  of 
either  pilot  team  suggests  transfer  of  control.  In  both  instances  the 
nonflying  pilot  (captain)  gives  the  Approach  Briefing;  also  the  Passen¬ 
ger  Arrival  Announcement  is  skipped  altogether.  Such  behavior  should 
constitute  a  basis  for  explicitly  requiring  the  Control  Transfer  pro¬ 
cedure  in  conjunction  with  other  SOPs  or  at  prespecified  points  in  a 
normal  flight. 

Three  other  pilot  teams  have  at  least  one  instance  each  of  failure 
to  verbally  accomplish  the  second  half  of  a  "double  transfer."  In  each 
case  the  flying  pilot  is  the  captain.  It  is  conceivable  that  some  sort 
of  visual  sign  is  substituted  for  the  prescribed  statements,  but  that 
would  still  be  noncompliance.  Since  the  observers  do  not  describe  any 
visible  transfer  activity,  the  true  state  of  the  system  must  be 
hypothesized.  Two  major  assumptions  are  logical:  (1)  sometimes  both 
pilots  think  they  have  control;  and  (2)  sometimes  neither  pilot  thinks 
he  has  control.  The  safety  Implications  of  either  assumption  are 
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indefensible.  The  prescribed  procedural  dialogue  succinctly  eliminates 
these  potentially  dangerous  situations. 

In  summary,  the  procedures  for  checklist  initiation/completion, 
parameter  callout,  configuration  change,  and  responsibility  transfer 
dictate  verbal  behaviors  that  can  reasonably  be  expected  to  enhance 
crew  coordination  and  flight  safety.  Except  in  the  case  of  Altitude 
Callouts,  noncompliance  appears  to  depend  more  on  the  operator(s) 
involved  than  on  the  requirements  of  the  procedures.  Further  support 
for  this  hypothesis  is  developed  in  the  next  section. 

Individuals  and  Crews 

Now  that  behavior  patterns  relative  to  specific  SOPs  have  been 
discussed,  it  is  time  to  consider  within-crew  procedural  compliance. 

The  perspective  of  the  foregoing  examination  disclosed  that  although 
some  existing  crew  coordination  procedures  could  be  improved,  their 
purposes  are  noble,  and  their  demands  are  generally  realizable.  Hence, 
the  following  paragraphs  treat  noncompliance  as  a  shortcoming  by  one  or 
more  operators.  The  extremes  of  observed  behavior  for  both  individuals 
and  crews  receive  principal  attention. 

a.  The  Upper  Levels  of  Compliance 

None  of  the  experimental  crews  demonstrate  totally  compliant 
behavior  across  all  three  operators  and  the  sampled  internal  coordina¬ 
tion  procedures.  Every  crew  does  have  at  least  one  member  who  conforms 
to  prescribed  verbal  behavior  with  high  consistency,  but  each  crew  also 
has  at  least  one  operator  who  regularly  violates  a  coordination  impera¬ 
tive.  Examples  of  compliance  by  one  crew  member  and  noncompliance 
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by  a  partner  are  prevalent  in  all  tasks  which  involve  two-way  communi¬ 
cations. 

With  respect  to  pilots  and  checklist  initiations,  for  example, 
the  captains  of  experimental  runs  five  and  fifteen  properly  command 
the  start  of  all  eight  challenge  and  response  checklists.  In  addition, 
these  individuals  uniformly  comply  with  the  verbal  requirements  for 
initiating  configuration  changes  and  transferring  aircraft  control. 
Their  copilots  on  the  other  hand,  frequently  fail  to  confirm  checklist 
commands  with  the  required  announcement;  and  the  copilot  on  run  five 
fails  to  confirm  the  "gear  down"  command. 

A  partial  reverse  image  of  the  pilot  teams  just  described  is  the 
one  participating  in  run  number  six.  The  copilot  of  that  crew  is  the 
only  pilot  checklist  challenger  who  announces  all  five  of  his  check¬ 
lists.  No  other  copilot  announces  more  than  two.  Regrettably,  the 
copilot  on  run  six  does  not  maintain  his  singularly  high  level  of 
performance  in  other  areas,  most  notably  control  of  airspeed,  angle  of 
attack,  and  sink  rate,  and  the  captain's  behavior  is  nonstandard  in 
many  respects  including  checklist  commands.  In  fact,  that  pilot  team 
has  the  worst  composite  level  of  compliance  observed.  Their  problems 
are  detailed  in  the  section  entitled  The  Lower  Levels  of  Compliance. 

In  the  meantime,  proper  behavior  merits  further  attention. 

Perfect  adherence  to  SOPs  is  certainly  not  an  impossibility.  The 
flight  engineers  on  runs  three,  four,  and  five  each  execute  the 
sampled  procedures  flawlessly;  but,  as  before,  their  associates  are 
less  than  perfect.  Copilot  noncompliance  on  run  five  has  already  been 
mentioned.  On  runs  three  and  four  the  captains  as  well  as  the  co¬ 
pilots  give  multiple  demonstrations  of  nonstandard  coordination 
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behavior.  Checklist  init iat ion/completion  requirements  are  not  fully 
met  by  either  crew,  and  run  three  is  plagued  by  incomplete  communica¬ 
tions  regarding  configuration  changes.  Clearly,  perfect  procedural 
compliance  by  a  single  operator  does  not  guarantee  the  same  kind  of 
behavior  by  the  remainder  of  the  crew. 

In  terms  of  adherence  to  the  crew  coordination  SOPs,  the  best 
overall  performance  occurs  on  run  number  ten.  This  is  essentially  an 
objective  appraisal.  Compliance  counts  are  generally  among  the  best 
observed  in  every  category. 

The  captain  misses  only  one  checklist  command,  and  all  checklist 
completion  statements  are  accomplished.  The  only  callout  omission  is 
induced  by  the  simulator  controller.  Coordination  of  configuration 
changes  is  sound  despite  landing  with  flaps  25  instead  of  flaps  30  on 
a  short  runway.  Lastly,  more  control  transfers  are  accomplished  than 
by  any  other  crew. 

During  run  ten,  as  in  runs  five  and  fifteen,  the  captain's  aware¬ 
ness  of  the  demands  on  his  subordinates  combined  with  his  assertiveness 
in  directing  joint  activities  appears  to  have  a  positive  influence  on 
crew  cohesion  and  procedural  compliance.  Even  though  none  of  these 
captains  is  procedurally  perfect,  each  maintains  a  high  standard. 
Collectively,  their  attention  to  the  fine  points  of  crew  coordination 
procedures  sets  them  apart  from  their  peers.  In  a  very  real  sense 
they  exhibit  a  positive  form  of  leadership  by  example. 
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b.  The  Lower  Levels  of  Compliance 

Among  the  lower  levels  of  personal  and  group  compliance  with  SOPs, 
attention  to  detail  by  the  pilot-in-command  is  anything  but  exemplary. 
Each  of  the  operators  in  run  number  eight  exhibits  very  selective  com¬ 
pliance  with  crew  coordination  procedures.  The  captain  is  typical.  He 
omits  the  first  lour  checklist  commands  but  does  not  miss  a  single 
configuration  change  instruction.  Similarly,  the  copilot  omits  every 
checklist  announcement  but  confirms  every  configuration  change.  In 
terms  of  quality  and  quantity  the  crew's  use  of  verbal  coordination 
procedures  is  questionable,  but  there  is  no  question  about  who  is  in 
command.  The  captain  sets  the  pace  of  crew  activity  and  regularly 
rechecks  the  status  of  various  tasks. 

The  same  cannot  be  said  for  runs  six,  twelve,  and  thirteen.  In 
all  outward  respects,  the  pre-experiment  characteristics  of  these 
three  crews  resemble  those  of  other  crews  in  the  study.  However,  for 
unknown  reasons,  the  copilot  becomes  the  flying  pilot  on  the  Dulles- 
JKF  segment.  In  each  instance  the  captain  neglects  to  explicitly 
allocate  command  authority  to  the  copilot. 

All  three  crews  experience  some  breakdown  in  internal  communica¬ 
tions.  Uncertainty  about  who  should  command,  announce,  and  challenge 
the  first  five  checklists  is  one  indication  of  fundamental  confusion 
over  proper  roles.  The  most  pronounced  rivalry  for  leadership  and 
the  most  serious  failure  to  exercise  command  responsibility  both  occur 
during  experimental  run  number  six. 

The  copilot  on  that  run  attempts  to  exert  leadership  by  announc¬ 
ing  and  then  immediately  reading  the  challenges  of  the  Start,  Pre-taxi, 
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Taxi,  and  Takeoff  Checklists.  He  does  this  without  a  command  from  the 
captain,  and  he  actually  tries  to  accomplish  the  tasks  of  the  flying 
and  nonflying  pilots  simultaneously.  Between  the  start  of  the  takeoff 
roll  and  level  off  at  cruising  altitude  crew  coordination  shows  signs 
of  very  serious  deficiencies.  The  captain  misses  two  Takeoff  Callouts. 
All  three  members  contribute  to  a  low  altitude  stick  shaker  activation 
(a  near  stall  condition).  Both  pilots  fail  to  verbalize  the  last  incre¬ 
ment  of  Flap  Retraction  (from  flaps  1  to  flaps  up) ,  and  the  copilot 
issues  three  separate  orders  to  the  engineer  to  do  the  After  Takeoff 
Checklist  before  any  form  of  acknowledgement  is  offered. 

At  no  time  during  flight  does  the  copilot  suggest  or  initiate 
transfer  of  control  to  the  captain.  Regardless,  the  copilot's  attempt 
at  physical  as  well  as  symbolic  leadership  is  ultimately  supplanted  by 
the  captain's  command  to  the  flight  engineer  to  begin  the  Descent 
Checklist.  That  action  serves  as  an  omen  of  further  deterioration  in 
aircraft  control  and  crew  management.  The  Approach  Radio  Checks  are 
neither  properly  commanded  nor  properly  completed.  The  copilot 
preempts  the  captain  on  the  first  two  Approach  Callouts  during  a  poorly 
executed  ILS  approach  which  finally  ends  in  a  go-around  (following  a 
second  outburst  from  the  Ground  Proximity  Warning  System).  The  cap¬ 
tain  fails  to  intervene  when  prescribed  flight  parameters  are  exceeded, 
and  on  the  second  ILS  approach  he  fails  to  callout  "minimums."  The 
pilot  team's  substandard  behavior  is  culminated  by  the  captain's 
omission  of  all  mandatory  callouts  during  the  landing  roll. 

No  other  crew  has  a  pattern  of  procedural  noncompliance  as  broad 
or  as  deep  as  the  one  just  described.  However,  leadership  and  crew 
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coordination  problems  are  evident  in  each  of  the  two  other  crews  with  a 
flying  copilot.  On  run  number  twelve  t  lie  captain  directs  the  copilot 
to  start  three  of  the  first  four  checklists.  He  also  commands  the 
engineer  to  begin  the  After  Takeoff  Checklis; .  Meanwhile,  the  copilot 
performs  nonflying  duties  and  reads  all  appropriate  challenges  in  the 
first  five  checklists.  The  copilot  does  not  formally  announce  any  of 
the  checklists,  and  his  only  formal  completion  statement  accompanies 
the  Pre-start  Checklist.  Twice  prior  to  takeoff  the  flight  engineer 
tells  the  pilots  that  he  is  unprepared  to  assume  EGT  monitoring  responsi¬ 
bilities.  After  his  second  refusal  no  crew  member  makes  further  men¬ 
tion  of  that  Transfer  Procedure. 

The  captain  intermittently  performs  leadership  functions  in 
flight,  especially  when  the  copilot  appears  "overloaded"  or  "behind 
the  aircraft."  For  example,  the  captain  advises  the  passengers  of 
turbulence,  he  conducts  the  Approach  Briefing  during  descent;  and  he 
requests  the  go-around  EPR  setting  from  the  engineer.  All  of  these 
things  are  normally  accomplished  by  the  flying  pilot.  The  copilot  does 
manage  to  call  for  initiation  of  the  Descent,  Approach,  and  Landing 
Checklists;  but  he  does  so  hesitatingly  without  much  conviction  or 
self-confidence.  He  seems  to  be  caught  between  two  cockpit  roles, 
leader  and  follower. 

Neither  pilot  ever  suggests  the  desirability  of  transferring  con¬ 
trol  of  the  aircraft.  The  one  fact  of  which  the  copilot  seems  certain 
is  that  he  is  flying  the  airplane,  and  he  does  nothing  to  change  that. 
Although  animosity  is  not  apparent  and  cooperation  between  the  pilots 
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does  occur,  their  specific  communications  tend  to  inhibit  rather  than 
enhance  crew  coordination. 

When  interpretations  and  expectations  regarding  command  authority 
and  responsibility  are  not  consistent  among  crew  members,  uncertainty 
and  confusion  may  preclude  any  sense  of  satisfaction  in  procedural 
compliance.  The  pilots  on  run  twelve  seem  unsure  of  who  should  pro¬ 
vide  basic  leadership. 

In  run  thirteen  leadership  and  compliance  with  SOPs  take  on  two 
new  dimensions.  First,  the  captain  informs  the  engineer  that  the  two 
pilots  have  flown  together  before,  and  they  know  each  other’s  style. 

At  the  same  time  he  advises  the  FE  to  freely  question  any  pilot  activ¬ 
ity  that  does  not  appear  prudent.  Second,  the  captain  actually  performs 
some  of  the  checklist  duties  of  a  nonflying  pilot.  In  particular,  he 
reads  the  challenges  for  the  Pre-taxi,  Taxi,  and  Takeoff  Checklists. 

For  the  Taxi  Checklist  the  copilot  even  issues  an  initiation  order  to 
his  nominal  supervisor.  Throughout  the  mission  the  two  pilots  work  in 
fairly  close  cooperation.  Although  their  compliance  with  crew  coor¬ 
dination  procedures  is  far  from  impeccable,  they  freely  discuss  oper¬ 
ational  issues  and  responsibilities.  They  formally  transfer  control  of 
the  aircraft.  They  seem  to  enjoy  sharing  leadership  and  support 
activities . 

In  contrast,  the  flight  engineer  seems  like  an  outsider.  He  does 
not  act  like  a  full  fledged  member  of  the  crew.  He  never  explicitly 
takes  responsibility  for  EGT  monitoring.  He  does  not  offer  any  informa¬ 
tion  about  the  status  of  the  After  Takeoff  Checklist  until  after  the 
copilot's  second  request.  He  initiates  the  Descent  Checklist  without 
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any  direction  from  the  pilots,  and  finally  he  fails  to  respond  to  the 
first  order  to  commence  the  Approach  Checklist.  None  of  the  engineer's 
behavior  blatantly  degrades  flight  safety,  but  the  observed  deficien¬ 
cies  clearly  suggest  the  need  for  improvement  in  total  crew  coordina¬ 
tion.  Ascribing  the  engineer’s  noncompliance  to  the  presence  of  a 
flying  copilot  or  to  a  particular  pairing  of  pilots  is  problematical. 
Nevertheless,  the  absence  of  a  single  identifiable  leader  on  any  of 
the  three  crews  just  described  is  cause  for  concern  about  crew  member 
perceptions  and  compliance  with  procedures  whenever  a  copilot  is  at 
the  helm. 

Categories  of  Operation  Error 

The  instances  of  procedural  noncompliance  given  above  are  imputed 
to  one  or  more  individual  operators .  This  is  in  contrast  to  the 
amorphous  assignment  of  diverse  errors  to  entire  crews  in  the  original 
NASA  study  (cf.  Chapter  II).  In  the  qualitative  stages  of  analysis 
Dr.  Ruffell  Smith  and  his  colleagues  pay  scant  attention  to  the 
identity  of  the  flying  pilot  or  to  the  perpetrator(s)  of  error.  The 
data  they  collected  permit  somewhat  greater  specificity  than  they 
employed. 

Most  of  the  errors  tabulated  in  the  original  report  have  been 
reverified  and  affiliated  with  the  appropriate  crew  member(s). 

Appendix  D  contains  a  complete  listing  of  errors  by  run  number, 
original  category,  descriptive  phrase,  and  applicable  operator(s) . 

Two  of  the  reported  errors  could  not  be  substantiated.  A  Flying  error 
for  run  ten  was  not  supported  by  any  available  documentation,  and  a 
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Navigation  error  listed  for  the  first  segment  of  run  thirteen  actually 
occurred  on  the  second  segment  of  that  mission.  In  addition,  three 
previously  unrecorded  Communication  errors  were  detected  and  logged, 
one  each  on  run  numbers  three,  ten,  and  twelve. 

This  author's  review  of  the  experimental  protocols,  the  audio 
data,  the  aircrew  manuals,  and  Dr.  Ruff  ell  Smith's  notes  has  led  to 
the  updated  error  summary  in  Table  20.  The  new  categories  of  error 
refer  directly  to  the  responsible  operator(s)  :  pilot  flying  (PF) , 
pilot  not  flying  (PNF) ,  captain,  copilot,  pilot  team,  flight  engineer 
(FE) ,  and  entire  crew.  Other  combinations  of  operators,  such  as 
PF/FE  or  captain/FE  are  possible;  but  no  errors  are  recorded  in  any 
such  category. 

A  synopsis  of  the  errors  attributable  to  different  operators 
discloses  that  flying  pilots,  nonflying  pilots,  captains,  and  copilots 
have  similar  numbers  of  total  errors  (20,  21,  20,  and  21  respectively). 
Naturally  enough,  PF  errors  tend  to  involve  manual  control  activities 
such  as  improper  use  of  the  autopilot  or  the  flight  director  and  fre¬ 
quent,  abrupt  throttle  movements  during  the  approach  to  landing.  The 
PNF  errors  encompass  most  of  the  communications  and  navigation  miscues 
listed  in  the  NASA  report.  Aircraft  or  ground  station  call  sign  prob¬ 
lems  and  radio  tuning  difficulties  comprise  the  majority  of  PNF  mis¬ 
takes.  Seventeen  of  the  twenty-one  PNF  errors  relate  to  communicat:  ■ 
or  navigation  activity. 

Are  errors  by  the  PF  more  serious  than  those  of  the  PNF'' 
context  of  the  experimental  scenario  the  answer  would  hav. 
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"yes."  In  actual  flight  operations  the  answer  is  a  more  equivocal 
"probably."  In  the  simulator  neither  the  PNF  nor  the  "air  traffic 
controller"  is  overly  sensitive  to  nonstandard  communication  or 
navigation  practices.  No  other  aircraft  will  be  in  the  simulator's 
"airspace,"  and  no  violations  of  the  FARs  will  be  forthcoming.  In 
contrast,  the  PF  must  be  genuinely  concerned  about  aircraft  performance. 
Although  simulated  mid-air  collisions  may  not  be  plausible,  unintended 
contacts  with  the  "ground"  certainly  are. 

At  present  it  is  impossible  to  say  whether  pilots  are  more  or 
less  prone  to  err  in  the  simulator  than  they  are  in  operational 
flight.  Errors  are  known  to  occur  in  both  environments.  Dr.  Ruffell 
Smith,  for  one,  believes  that  his  composite  error  rates  are  comparable 
to  those  observed  aboard  a  European  short-haul  airline  during  routine 
operations.  (53:19-20) 

Thus  far,  only  errors  imputed  to  a  single  crew  member  have  been 
discussed.  Many  significant  errors  involve  more  than  one  operator. 
Attention  is  first  given  to  those  implicating  the  pilots  jointly. 

Every  crew  has  at  least  one  pilot  team  error.  Aircraft  control, 
equipment  operation,  and  Air  Traffic  Control  procedures  are  violated. 

The  verified  deficiencies  include  early  retraction  of  flaps  on  take¬ 
off,  improper  pairing  of  the  Flight  Director  computers,  exceeding 
speed/altitude  tolerances,  failing  to  use  the  Turbulence  mode  of  the 
autopilot  and  selecting  the  wrong  navaid.  Some  of  these  errors  appear 
to  be  associated  with  distraction,  inaccurate  perception,  ignorance, 
or  lack  of  personal  motivation;  but  causal  factors  are  impossible  to 
assign. 
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In  some  situations  one  pilot  verbally  acknowledges  awareness  of 
the  other's  act  or  intent  (e.g.,  flap  retraction  and  computer  selec¬ 
tion).  In  other  cases  silence  prevails.  A  reticent  pilot  could  be 
unaware  of  his  partner's  error;  he  could  be  giving  tacit  approval;  he 
could  be  fearful  of  "making  waves";  or  he  could  be  uncertain  and 
therefore  unwilling  to  expose  his  own  lack  of  knowledge.  At  any  rate, 
errors  that  are  either  ratified  or  undisputed  are  reinforced  and  are 
likely  to  recur  in  similar  circumstances.  The  degree  to  which  the 
observed  nonflying  airline  pilots  monitor  and  question  the  behavior 
of  their  comrades  is  certainly  less  than  that  envisoned  in  company 
policy. 

The  same  can  be  said  of  crews  as  a  whole.  The  errors  attributed 
to  an  entire  crew  display  the  same  shortage  of  awareness,  knowledge, 
and/or  initiative  that  is  seen  In  pilot  team  errors.  Of  the  ten 
experimental  runs,  only  one  has  no  verifiable  crew  errors  (run  number 
four) .  A  total  of  four  crews  use  the  wrong  flap  configuration  for 
landing,  and  a  fifth  uses  the  proper  configuration  but  for  the  wrong 
reason.  Two  separate  crews  fail  to  turn  on  anti-icing  systems  prior 
to  takeoff  and  then  compound  their  problems  with  power  reduction 
during  departure.  Another  crew  omits  the  anti-ice  step  of  the  turbu¬ 
lence  penetration  procedure.  If  captains  cannot  be  counted  on  to 
supervise  their  subordinates'  activities  and  if  copilots  and  engineers 
are  too  preoccupied  to  assist  their  leader  when  he  errs,  then  flying 
may  have  become  too  complicated. 

Perhaps  there  are  too  many  procedures  for  the  average  crew  member 
to  know  and  employ.  The  Normal  Operating  Procedures  presented  in 
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Chapter  III  are  augmented  by  literally  scores  of  unique  instrument 
approach  procedures,  aircraft  operating  limitations,  and  airport 
restrictions  as  well  as  the  aforementioned  Alternate,  Abnormal, 
Irregular,  and  Emergency  procedures. 

The  selected  crew  coordination  procedures  constitute  a  small  but 
important  portion  of  a  crew  member's  procedural  inventory.  How  he 
behaves  in  relation  to  those  few  SOPs  may  be  indicative  of  his  pro¬ 
pensity  for  involvement  in  other  types  of  operating  procedure  error. 

Mathematical  Modeling 

Trying  to  investigate  potential  relationships  between  discrete, 
enumerative  error  data  and  discrete,  enumerative  compliance  data  is  a 
challenge  under  ideal  conditions.  In  this  research  the  scarcity  of 
usable  data  has  made  conditions  far  from  ideal.  From  the  beginning, 
all  types  of  error  and  noncompliance  were  necessarily  assumed  to  be  of 
equal  criticality. 

At  first  blush  a  supposition  of  this  sort  seems  rash.  It  equates 
a  higher  than  prescribed  angle  of  attack  to  omission  of  the  aircraft 
call  sign  during  a  radio  transmission.  These  are  vastly  different 
errors;  however,  the  criticality  of  each  is  truly  situation  dependent. 
A  high  angle  of  attack  in  smooth,  clear  air  with  airspeed  well  above  a 
stall  is  hardly  critical;  but  the  3ame  angle  of  attack  at  a  low  alti¬ 
tude  and  airspeed  could  be  critical  indeed.  When  air  traffic  and  ATC 
radio  communications  are  dense,  even  a  missed  call  sign  can  be 
dangerous.  In  other  words,  there  is  no  convenient  scale  for  ranking 
operator  errors  independent  of  unknown  state  variables. 


■  —  <  lb 
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A  parallel  predicament  applies  to  the  evaluation  of  noncompliance 
with  crew  coordination  procedures.  In  some  situations,  for  example  the 
pre-takeoff  rush  In  the  simulator  flight,  omission  of  a  checklist  com¬ 
mand  or  announcement  may  seem  justified;  but  Identifying  all  the 
combinations  of  state  variables  which  might  vindicate  noncompliance  is 
outside  the  scope  of  this  project.  Consequently,  diverse  deviations 
in  crew  coordination  (omissions,  preemptions,  tardiness,  etc.)  are 
treated  equivalently. 

Aggregating  behavioral  variables,  whether  they  are  error  counts 
or  compliance  counts,  eliminates  a  difficult  scaling  problem  and  a 
profusion  of  low  frequency  classifications.  However,  the  relatively 
small  size  of  the  experimental  data  base  also  creates  modeling  problems. 
Discrete  multivariate  techniques  are  not  suitable  because  cell  fre¬ 
quencies  would  be  too  small  for  the  statistical  tests  to  have  satisfac¬ 
tory  power. 

One  mathematical  model  which  can  be  used  with  a  limited  number  of 
observations  is  the  Poisson  probability  distribution.  It  has  been 
employed  in  analyses  of  human  behavior  (65)  as  well  as  product 
reliability  (27).  The  underlying  assumption  of  the  model  is  that 
errors  occur  randomly  over  time.  If  this  assumption  is  true,  then  the 
number  of  errors  observed  conforms  to  a  Poisson  distribution.  In 
standard  notation  the  Poisson  probability  of  experiencing  m  errors 
during  a  time  interval  of  length  t  is  given  by 

p(m,t)  ■  [exp(-At) ]  •  t At  J™  /m! ,  m^O,!^ , . . . ; 


A  >0,  t  >  0 
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In  Che  simulated  flight  mission  the  time  period  is  considered  to 
be  approximately  the  same  for  each  experimental  run  (t“l)  so  the 
probability  function  simplifies  to 

p(m)  =  [exp(-X) ]m/m! ,  m=0,l,2,...;  X  >  0 

The  only  parameter  which  must  be  determined  to  permit  testing  of  this 
model  is  X,  the  mean  of  the  distribution. 

Estimation  of  the  single  parameter  X  permits  testing  of  the 
randomness  assumption.  In  fact,  randomness  can  be  tested  even  when  X 
is  different  for  each  trial  of  an  experiment  as  long  as  the  parameter 
is  treated  as  a  function  of  one  or  more  variables  which  can  be  measured 
in  every  trial.  The  calculations  herein  assume  that  the  particular  X 
applicable  to  an  individual  operator,  pilot  team,  or  crew  can  be  com¬ 
puted  as  a  linear  function  of  the  procedural  compliance  counts  developed 
earlier  in  this  chapter.  The  specific  linear  functions  to  be  used  are 
determined  by  regression  analysis. 

The  appropriateness  of  the  Poisson  distribution  for  error  counts 
is  examined  by  means  of  a  test  statistic  Z,  where  z  **  (m  -  mj/m-  with 
m  the  actual  number  of  errors  observed  and  m  the  number  predicted  by 
the  regression  equation.  If  the  distribution  of  m  is  Poisson,  then  the 
distribution  of  Z  will  be  approximately  normal  with  mean  zero  and 
variance  one.  Departures  from  normality  can  be  visualized  by  plotting 
Z  on  probability  paper.  This  type  of  analysis  is  justifiable  whenever 
X  is  large  enough  that  the  corresponding  Poisson  distribution  is  not 
severely  skewed.  In  this  study  the  smallest  mean  error  count  (£)  is 
1.1,  and  the  mean  values  of  greatest  interest  lie  between  2.0  and  10.9; 


hence,  the  test  just  described  is  suitable.  A  complete  list  of  the 
dependent  variables  of  Interest  and  their  observed  means  is  shown  in 
Table  21. 


TABLE  21 

DEPENDENT  VARIABLE  CATEGORIES  AND  MEANS 
(Error  Counts) 


*Category 

Code 

Mean  (X) 

PF  alone 

PF 

2.0 

Captain  alone 

CAP 

2.0 

PNF  alone 

PNF 

2.1 

Copilot  alone 

COP 

2.1 

Pilot  team 

PTM 

2.3 

PF  +  team 

PTM 

4.3 

CAP  +  PTM 

TCAP 

4.3 

PNF  +  PTM 

TPNF 

4.4 

COP  +  PTM 

TCOP 

4.4 

Entire  crew 

CRW 

1.1 

PF  +■  PTM  +  CRW 

CPF 

5.4 

CAP  +  PTM  +  CRW 

CCAP 

5.4 

PNF  +  PTM  +  CRW 

CPNF 

5.5 

COP  +  PTM  +  CRW 

CCOP 

5.5 

PF  +  PNF  +  2(PTM)  +  2(CRW) 

PLTS 

10.9 

*N ote:  Although  certain  error  categories  are  mutually  exclusive, 
e.g.,  PF  alone  and  PNF  alone,  others  are  not,  e.g.,  PF  alone  and 
Captain  alone.  Mutually  exclusive  categories  are  treated  as  inde¬ 
pendent  random  variables  and  combined  linearly  to  form  other 
meaningful  categories. 
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proportionate  reduction  of  the  total  variation  in  Y  associated  with  use 
of  the  set  of  variables  X^,  X£,  . ..,  Xn«  Both  the  simple  and  multiple 
coefficients  of  determination  must  lie  in  the  interval  [0,1],  The 
larger  the  value,  the  more  the  total  variation  in  the  dependent 
variable  is  reduced  by  introduction  of  the  independent  variable(s) . 

When  regression  models  are  used  for  descriptive  purposes,  as  they  are 
here,  there  is  no  implication  of  causality  between  the  independent  and 
dependent  variables.  A  coefficient  of  determination  close  to  unity 
indicates  a  relatively  high  degree  of  association  between  a  specific 
set  of  Xs  and  a  particular  Y,  but  the  Xs  do  not  produce  the  Y. 
Elucidation  on  the  theory  and  application  of  numerous  statistical 
models  is  contained  in  an  excellent  treatise  by  Neter  and  Wasserman 
(46). 

b.  Independent  Variables 

The  verbal  behaviors  previously  illustrated  in  Tables  13  through 
19  have  been  quantified  by  straightforward  enumeration.  This  simplis¬ 
tic  approach  has  two  salient  limitations,  but  it  does  yield  a  set  of 
independent,  predictor  variables  some  of  whose  elements  look  remarkably 
like  the  error  counts  above. 

The  first  limitation  has  to  do  with  the  range  of  each  variable, 
that  is  the  difference  between  the  maximum  and  minimum  values  each 
could  possibly  assume.  For  example,  the  verbal  interchange  required 
to  accomplish  the  Transfer  of  EGT  Monitor  responsibility  has  to  occur 
only  once  during  each  mission  segment.  It  is  supposed  to  be  initiated 
by  the  flight  engineer  and  concluded  by  the  captain.  Thus,  the 
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behavioral  variable  is  binary  for  both  operators.  Since  no  other  SOP 
prescribes  a  comparable  interchange  between  the  same  two  crew  members 
this  single  data  point  predictor  has  limited  utility  in  characterizing 
routine  behavior. 

The  second  limitation  on  activity  counts  is  variability.  The 
variance  of  a  random  variable  is  restricted  by  its  range.  If  counts 
of  procedural  activity  show  little  variability  (even  when  the  possible 
range  is  large) ,  then  prediction  of  wider  performance  gradations  may 
be  impossible. 

Among  crew  coordination  SOPs,  the  configuration  change  interactions 
between  pilots  and  the  checklist  completion  statements  required  of 
engineers  produce  behavioral  variables  with  too  little  variation  to 
suggest  distinctions  among  the  subjects.  Eight  of  ten  flying  pilots 
do  not  miss  a  single  configuration  change  request;  the  others  miss  one 
and  two  respectively.  The  engineers  are  similarly  uniform  with  respect 
to  checklist  completions.  Two  FEs  omit  one  statement  each;  all  the 
rest  comply  perfectly. 

Despite  the  limitations,  or  insensitivity,  of  many  crew  coordina¬ 
tion  variables,  there  are  several  aspects  of  pilot  behavior  which  have 
potential  as  predictor  variables.  These  indicators  possess  moderate 
range  and  variability.  They  are  listed  along  with  two  variables 
displaying  crew  member  experience  in  Table  22. 

The  first  three  behavioral  variables  in  the  table  deal  with  the 
initiation  of  normal  challenge  and  response  checklists.  Eight  command 
opportunities  are  available  for  the  flying  pilot  in  each  data  run 
(PFCK) ;  the  number  of  occasions  of  procedurally  compliant  behavior  is 
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recorded.  Correspondingly,  there  are  five  checklist  announcement 
opportunities  of  pilots  (PA)  and  three  for  flight  engineers  (FEA) . 
Again,  instances  of  procedural  compliance  are  counted. 

Another  type  of  pilot  behavior  which  displays  variability  among 
the  nonflying  pilots  concerns  the  existence  and  timeliness  of  mandatory 
callouts.  This  particular  indicator  (PNFC)  represents  the  number  of 
prescribed  callouts  completed  by  the  appropriate  pilot  (the  PNF)  at  the 
proper  time.  It  makes  no  allowance  for  "late"  altitude  callouts 
(accomplished  after  the  automatic  mechanical  alert  at  900  feet  from 
target  altitude)  or  for  callouts  accomplished  by  the  flying  pilot. 
Seventeen  on-time  callouts  are  possible;  however,  sixteen  is  the  larg¬ 
est  observed  value. 

An  aspect  of  procedural  compliance  which  has  no  mandated  or 
maximum  value  concerns  Transfer  of  Aircraft  Control  between  the  pilots. 
Implementation  of  this  optional  procedure  is  not  restricted  to  any 
particular  phase  of  a  mission  or  set  of  environmental  circumstances. 
Compliance  depends  solely  on  the  level  of  initiative,  cooperation, 
trust,  and  communication  residing  in  the  pilots. 

Each  complete  "double  transfer"  of  control  is  counted  as  two  occa¬ 
sions  of  compliance  with  the  transfer  procedure  (TRAN) .  Each  partial 
or  incomplete  "double  transfer"  counts  as  one  instance  of  compliance. 
Across  the  ten  pilot  teams  the  average  number  of  transfers  is  2.8  and 
the  sample  standard  deviation  is  1.87.  Significantly,  though,  two 
crews  have  no  transfer  whatsoever  while  the  crew  on  run  ten  actually 
completes  six  transfers. 
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Two  additional  Independent  variables  which  are  not  direct  measures 
of  internal  coordination  but  are  believed  to  capture  a  material 
feature  of  crew  composition  have  to  do  with  flying  experience  in  the 
B-747  aircraft.  Since  crew  members  must  accept  assignment  to  differ¬ 
ent  airframes  as  part  of  career  progression,  experience  within  a  crew 
can  vary  widely.  Accumulation  of  1000  flying  hours  in  a  given  aircraft 
is  an  arbitrary  reference  point  for  experience-in-type.  The  total 
number  of  crew  members  (CREX)  and  the  total  pilots  (PEX)  with  more 
than  1000  hours  each  in  the  B-747  are  the  chosen  variables.  For  an 
average  pilot  who  flies  600  to  700  hours  per  year  this  equates  to 
approximately  one  and  one-half  years  of  line  flying  experience.  No 
crew  member  participating  in  the  experiment  has  fewer  than  5000  total 
flying  hours,  but  many  have  less  than  1000  hours  in  the  747. 

The  NTSB,  the  FAA,  and  individual  airlines  have  at  various  times 
noted  checklist  behavior,  callout  performance,  transfer  of  aircraft 
control,  and  experience-in-type  as  factors  which  are  associated  with 
airline  mishaps.  The  statistical  relationships  between  those  factors 
as  quantified  here  and  other  manifestations  of  human  performance  are 
highlighted  below. 

c.  Correlations 

A  frequent  objective  of  descriptive  regression  analysis  is  to 
account  for  a  sizable  percentage  of  the  variability  in  a  dependent 
variable  using  as  few  independent  variables  as  possible.  In  this 
study  corroboration  of  operator  error  using  a  maximum  of  two  predicators 


is  desired. 
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Coefficients  of  determination  have  been  computed  for  each  of  the 
fifteen  categories  of  operator  error  using  each  of  the  seven  independ¬ 
ent  variables  alone,  in  pairs,  and  in  triples.  The  maximum  values  and 
corresponding  variables  are  shown  in  Table  23.  All  calculations  were 
accomplished  with  the  aid  of  the  RSQUARE  routine  in  the  Statistical 
Analysis  System  (SAS)  computer  package  (3:216). 

For  eight  of  the  dependent  variables  (PNF,  COP,  TCAP,  TPNF, 

TCOP,  CPNF,  CCOP,  and  PLTS)  the  flying  pilot’s  compliance  with  pro¬ 
cedure  in  issuing  checklist  commands  (PFCK)  is  the  single  best  predic¬ 
tor.  All  of  the  other  error  counts,  except  pilot  team  (PTM) ,  are  best 
predicted  by  the  nonflying  pilot's  callout  compliance  (PNFC) .  Only 
PTM  errors  and  entire  crew  errors  (CRW)  have  coefficients  of  deter¬ 
mination  less  than  0.5.  Interestingly,  the  PFCK  indicator  is  usually 
the  best  predictor  of  copilot  or  nonflying  pilot  error  counts  while 
the  PNFC  indicator  is  better  for  captain  and  flying  pilot  errors. 

When  two  predictor  variables  are  used,  PFCK  belongs  to  the  maxi- 
2 

mum  R  pair  for  all  dependent  variables  except  PTM  and  TPF.  PNFC 

occurs  in  eight  of  the  best  pairs.  The  PFCK  and  PNFC  combination  has 

2 

the  largest  value  of  R  for  seven  of  the  error  counts,  and  for  five 
2 

others  its  R  value  differs  from  the  maximum  by  less  than  0.08.  In 
most  cases  the  addition  of  a  third  independent  variable  adds  relatively 
little  to  the  largest  pairwise  coefficient  of  determination  while 
eliminating  another  degree  of  freedom  from  the  mean  square  error  calcu¬ 
lation  in  the  regression  model.  Consequently,  the  PFCK-PNFC  combina¬ 
tion  receives  primary  attention. 


TABLE  23 
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With  respect  to  the  dependent  variables  which  reflect  errors  by 
the  flying  pilot  (PF,  TPF,  CPF),  by  the  captain  (CAP,  TCAP,  CCAP) ,  and 
by  the  two  pilots  collectively  and  individually  (PLTS) ,  the  regression 
models  for  PFCK  and  PNFC  together  are  all  highly  significant  (p  <  .01) . 
The  coefficient  of  multiple  determination  is  never  less  than  0.741. 
Moreover,  the  STEPWISE  regression  routine  (3:251-6)  keeps  both  inde¬ 
pendent  variables  in  its  model  for  each  dependent  variable.  When  the 
same  two  predictors  are  regressed  with  the  other  error  counts,  none  of 
the  models  is  highly  significant.  In  fact,  STEPWISE  never  allows 
more  than  one  of  the  two  predictors  in  its  final  model.  In  no  case 
does  the  coefficient  of  multiple  determination  reach  0.7. 

Clearly,  the  best  two-variable  corroboration  of  Ruffell  Smith's 
crew  assessment  occurs  between  the  captain/flying  pilot  error  counts 
and  the  conformance  to  selected  crew  coordination  imperatives  by  the 
two  pilots.  Although  compliance  with  one  type  of  procedure  has  no 
mechanical  connection  to  compliance  with  another  type,  a  methodology 
for  more  detailed  description  of  operator  behavior  has  been  demon¬ 
strated.  Since  human  evaluation  of  human  performance  is  always  subject 
to  innumerable  fallacies  and  biases,  it  can  only  be  hoped  that  the 
simple  metrics  presented  here  reflect  more  complex  and  more  ambiguous 
attributes  which  cannot  be  readily  measured. 

d.  Error  Distributions 

The  final  issue  of  concern  in  this  assessment  of  aircrew  per¬ 
formance  is  the  appropriateness  of  the  Poisson  assumption  for  error 
counts.  In  two  of  the  error  categories  for  which  PFCK  and  PNFC  yield 
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highly  significant  regression  models,  namely  PF  and  CAP,  the  predicted 
number  of  errors  for  run  ten  happens  to  be  negative.  Since  an  error 
count  less  than  zero  is  an  impossibility,  the  previously  described 
test  cannot  be  applied  to  those  categories  of  error. 

No  comparable  problem  is  encountered  with  TPF,  TCAP,  CPF,  CCAP,  or 
PLTS,  error  counts  with  sizable  mean  values.  Their  test  plots  are 
shown  in  Appendix  F.  The  graphical  analysis  reveals  that  each  error 
category  other  than  PLTS  shows  considerable  deviation  from  the  ideal 
standard  normal  line,  especially  in  the  tails  of  the  distribution. 

This  could  indicate  that  the  errors  are  not  random  or  that  the  crew 
members  belong  to  more  than  one  identifiable  population.  On  the  other 
hand,  the  Poisson  assumption  cannot  be  dismissed  for  PLTS.  Its  sample 
data  points  conform  to  the  standard  normal  line  remarkably  well.  Since 
PLTS  is  a  linear  combination  of  four  variables  with  unknown  distribu¬ 
tions,  there  is  no  obvious  reason  why  it  should  be  Poisson  distributed. 
Although  the  plots  (Appendix  F)  are  not  conclusive  evidence  as  to  the 
distribution  of  any  of  the  error  counts,  they  suggest  that  some  other 
distributions  may  be  more  suitable  for  modeling  the  frequency  of  error 
in  operator  specific  categories. 

In  summary,  PFCK,  a  crew  coordination  indicator  representing  the 
flying  pilots'  initiative;  meticulousness,  and  ability  to  communicate 
in  a  leadership  position,  is  well  correlated  with  twelve  of  the 
fifteen  error  categories  studied.  Also,  the  PFCK-PNFC  duo  achieves 
excellent  results  in  predicting  captain,  flying  pilot,  and  total  pilot 
error  counts.  Despite  the  relatively  small  number  of  sample  data 
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points,  the  correlations  established  here  warrant  a  review  of  existing 
procedural  imperatives,  the  frequency  of  their  applicability  in  line 
operations  and  the  level  of  compliance  among  a  cross  section  of  major 
air  carriers. 


CHAPTER  V 


COCKPIT  MANAGEMENT  AND  AIRCREW  PROCEDURES 

The  behaviors  recorded  by  Ruff ell  Smith  emphasize  manual  control 
and  external  communication.  The  crew  coordination  behaviors  reported 
in  Chapter  IV  emphasize  verbal  interaction  among  the  cockpit  crew  mem¬ 
bers  themselves.  Both  views  tempt  an  observer  to  fixate  on  isolated 
events.  A  more  holistic  perspectiveon  human  performance  is  desirable 
and  beneficial. 

In  this  chapter  three  general  precepts  of  good  management  are 
defined,  and  their  applicability  to  airline  captains  is  discussed. 
Subjective  evaluations  of  managerial  style  in  the  simulator  are  then 
compared  to  the  principal  objective  measures  of  performance  used 
earlier.  Relationships  between  qualitative  and  quantitative  assess¬ 
ments  suggest  the  strengths  as  well  as  the  weaknesses  of  a  procedural 
compliance  perspective  on  cockpit  management. 

Management  Precepts 

The  study  of  management  philosophy  and  managerial  performance 
goes  on  continuously  in  many  organizations.  The  United  States  Air 
Force  is  an  excellent  example.  For  many  years  the  Air  Force  has  pub¬ 
lished  an  official  document.  Air  Force  Manual/Regulation  25-1,  which 
explains  universal  principles  and  processes  of  management.  Its  Influ¬ 
ence  on  this  author's  thinking  and  research  is  undeniable.  In  fact,  it 


107 


contains  one  particular  statement  which  concisely  describes  the  value 
of  procedures  for  all  levels  of  management;  to  wit, "procedures  .  .  . 
are  the  heart  of  an  operational  system;  they  give  direction  to  its  ef¬ 
fort,  coordinate  it  in  place  and  time,  and  determine  whether  it  will 
perform  in  line  with  predetermined  objectives."  (62:14)  This  asser¬ 
tion  presupposes  compliance  with  established  procedures,  but  realities 
of  human  behavior  are  not  overlooked.  Dealing  with  noncompliance  and 
with  its  perpetrators  is  treated  as  another  phase  of  managerial 
responsibility . 

The  management  guide  also  proffers  sixteen  fundamental  precepts 
which  have  been  "proved  by  broad  experience  to  contribute  to  good 
management."  (62:44)  To  some  degree  each  one  can  be  applied  to  cockpit 
management,  but  many  of  them  are  most  appropriate  in  relatively  large, 
stable  organizational  settings.  Because  of  the  limited  size  and  the 
transitory  nature  of  a  cockpit  organization,  this  author  considers 
three  of  the  precepts  to  be  more  pertinent  than  the  rest.  These  are 
continuity,  cooperation,  and  discipline. 

Continuity  is  defined  as  the  ability  to  "plan  and  organize, 
insofar  as  possible,  for  the  full  period  of  contemplated  operations." 
(62:45)  Among  pilots,  continuity  is  frequently  referred  to  as  the 
capability  to  "stay  ahead  of  the  aircraft."  Command  of  a  sophisticated 
jet  aircraft  demands  continuity,  but  some  captains  display  the  faculty 
to  a  much  greater  degree  than  do  others. 

Concerning  the  precept  of  cooperation,  the  Air  Force  says,  "to 
cooperate  is  to  render  active  aid,"  and  to  recognize  that  "the  organi¬ 
zation’s  coordinated  effort  is  greater  than  the  sum  of  the  individual 
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uncoordinated  efforts  of  its  components."  (62:45)  In  an  airline  cock¬ 
pit  tasks  are  often  multi-step  and  time-sensitive.  Many  tasks  overlap 
each  other  in  space  and  time.  Operational  safety  demands  cooperation. 
Captains  must  be  willing  to  aid  their  subordinates  and  to  accept  aid 
from  them  when  it  is  needed.  In  the  experimental  scenario  considerable 
variation  in  crew  cooperation  was  observed. 

Similarly,  the  discipline  displayed  by  different  captains  varied 
greatly.  The  precept  directs  managers  to  "establish  and  enforce 
directives  and  procedures  that  are  essential  to  orderly  accomplishment 
of  the  objective."  (62:45)  Thus,  dicipline  recognizes  the  imperfections 
in  all  mortals  and  stresses  the  manager's  responsibility  to  assure 
procedural  conformance.  Since  SOPs  are  prescribed  for  virtually  every 
flight  task  (objective),  discipline  is  an  essential  aspect  of  effective 
cockpit  management. 

Subjective  Assessments 

This  writer  applied  the  foregoing  definitions  to  each  captain's 
documented  management  efforts  in  the  simulation.  Individuals  were 
judged  as  personifying  each  precept  to  a  high,  moderate,  or  low  degree. 
The  evaluations  attempted  to  compare  the  captains  to  a  model  pilot-in¬ 
command  rather  than  to  their  peers  in  the  experiment.  The  author's 
judgment  was  based  upon  detailed  knowledge  of  the  experimental  scenario 
coupled  with  personal  flight  experience  in  similar  circumstances. 

The  orientation  was  global;  one  exceptional  example  of  performance, 
good  or  bad,  did  not  qualify  a  subject  for  an  extreme  rating.  The  judg¬ 
ments  reflected  in  Table  24  have  been  introduced  at  this  point  to  provide 
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QUALITATIVE  ASSESSMENT  OF  CAPTAINS'  MANAGEMENT  STYLES 


Experimental 

Run  Number 

Management  Precepts 

Continuity 

Cooperation 

Discipline 

3 

M 

L 

M 

4 

H 

M 

H 

5 

M 

H 

M 

6 

L 

L 

L 

8 

M 

H 

M 

10 

M 

M 

M 

12 

L 

L 

L 

13 

L 

H 

L 

14 

H 

H 

H 

15 

H 

H 

M 

H  =  High 
M  =  Moderate 
L  *  Low 


a  retrospective  estimation  of  each  captain's  managerial  profile.  The 
purpose  is  to  obtain  a  comprehensive  subjective  evaluation  which  could 
be  collated  with  more  objective  measures  of  performance.  Differences 
among  the  qualitative  and  quantitative  measures  delineate  the  likeli¬ 
hood  that  volunteers  for  the  experiment  should  not  be  thought  to  belong 
to  a  single  aircrew  population. 

If  cooperation  was  present  at  all,  it  was  often  abundant.  Half 
of  the  captains  were  regarded  as  highly  cooperative.  Typically,  they 
shared  external  communication  tasks  with  their  copilots,  helped 
accomplish  preflight  radio  checks,  and/or  discussed  their  intentions 
with  subordinates.  However,  among  three  of  the  commanders,  cooperation 
was  definitely  insufficient.  In  each  case  the  copilot  could  have  used 


assistance  which  was  not  forthcoming.  Two  flying  copilots  had  to  com¬ 
plete  nonflying  tasks  because  their  captains  did  not  assume  those 
duties.  A  third  copilot  was  repeatedly  interrupted  during  task  execu¬ 
tions  by  an  impatient,  ego-centric  captain. 

Neither  continuity  nor  discipline  received  as  many  high  marks. 
Distribution  of  continuity  grades  was  more  nearly  uniform,  but  this 
evaluator  was  struck  by  several  examples  of  poor  planning;  namely 
rushed  checklists,  missed  passenger  announcements,  and  incomplete 
analysis  of  the  route  or  the  instrument  approach. 

Of  all  the  managerial  qualities,  discipline  needed  the  most 
improvement.  If  subordinates'  shortcomings  were  recognized  at  all, 
they  rarely  drew  a  comment.  A  laissez  faire  form  of  management  seemed 
in  vogue  on  at  least  half  of  the  flights.  Nevertheless,  safety  was 
seriously  compromised  only  when  subordinates  were  not  as  competent  as 
the  captain  had  apparently  imagined  them  to  be. 

Eight  of  the  nine  "lov/'scores  were  awarded  to  three  captains. 

Every  one  of  the  three  was  the  nonflying  pilot  for  the  segment  in  ques¬ 
tion,  and  each  one  had  600  or  fewer  hours  as  a  pilot -in -comand  of  the 
B-747  aircraft.  Two  of  the  three  did  not  work  well  with  either  sub¬ 
ordinate.  The  third  was  very  congenial  with  his  crew,  but  none  of  the 
three  actively  enforced  procedures. 

The  subjective  evaluations  presented  have  been  pictorially  com¬ 
pared  to  behavioral  indicators,  experience  indicators,  and  error 
counts  defined  in  Chapter  IV.  The  individual  graphs  are  shown  in 
Appendix  G.  The  most  significant  findings  for  each  precept  are  sum¬ 
marized  In  the  following  paragraphs. 


Continuity  scores  appear  inversely  related  to  error  counts  for  the 
captains  but  unrelated  to  copilot  errors.  This  corresponds  with  the 
intuitive  notion  that  captains  who  "get  behind  the  aircraft"  commit 
more  errors.  All  of  the  high  and  low  continuity  scores  are  associated 
with  pilot  teams  that  have  relatively  low  experience-in-type  (PEX) . 
These  extremes  may  indicate  that  as  experience-in-type  accumulates, 
the  better  planners  feel  less  need  for  anticipation  of  events  while 
the  poorer  planners  learn  from  their  mistakes  and  improve. 

Cooperation  shows  no  obvious  relationship  to  error  counts, 
experience,  or  compliance  with  crew  coordination  procedures.  Although 
the  three  commanders  with  low  cooperation  scores  have  the  three  high¬ 
est  solo  error  counts  (CAP),  the  counts  of  copilot  (COP),  pilot  team 
(PTM) ,  and  crew  (CRW)  error  do  not  confirm  any  affiliation  to  cooper¬ 
ativeness.  Additionally,  adherence  to  checklist  initiation  (PFCK) , 
callout  (PNFC) ,  and  control  transfer  (TRAN)  procedures  has  no  appar¬ 
ent  connection  to  cooperation.  Establishing  a  spirit  of  fraternity 
in  the  cockpit  may  be  an  invitation  to  excessive  trust  as  well  as  a 
summons  for  mutual  support. 

A  linear  relation  with  negative  slope  exists  between  discipline 
and  captains'  errors,  but  no  similar  relation  is  apparent  for  disci¬ 
pline  and  copilot  errors.  The  extremes  of  the  discipline  scale  belong 
to  captains  with  low  experience-in-type.  As  time  in  the  aircraft 
builds,  discipline  as  well  as  continuity  may  tend  toward  a  middle 
ground. 

In  general,  these  managerial  evaluations  substantiate  the  lower 
levels  of  procedural  compliance  better  than  they  do  the  middle  or  upper 
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levels.  Pilot  performance  on  simulator  runs  six,  twelve,  and  thirteen 
appears  unsatisfactory  in  several  respects.  All  other  pilot  teams 
behave  suboptimally  at  specific  points  in  time  according  to  at  least 
one  of  the  available  metrics;  but  the  others'  deficiencies  are  not  as 
diverse  or  as  pervasive  as  those  of  the  three  teams  whose  copilot  is 
flying  the  experimental  segment. 

One  of  the  benefits  of  qualitative  assessments  of  cockpit  manage¬ 
ment  is  that  they  illuminate  the  advantages  and  disadvantages  of 
quantitative  measures.  The  quantitative  approach  to  procedural  com¬ 
pliance  meets  the  issue  of  frequency  head  on.  A  human  operator  can 
be  given  feedback  on  exactly  how  many  times  he  executes  specific 
tasks  in  accordance  with  SOPs.  A  large  numerical  data  base  would 
permit  computation  of  behavioral  norms  and  would  allow  statistical 
inferences  about  the  individual  and  population  performances  of  many 
crew  members.  However,  enumerative  data  and  statistics  do  not  convey 
information  about  the  circumstances  surrounding  an  instance  of  pro¬ 
cedural  noncompliance  or  about  the  criticality  of  recorded  activity. 
Both  of  these  considerations  are  certainly  relevant  to  system  safety. 
The  disparate  relationships  among  the  various  quantitative  and  qualita¬ 
tive  measures  confirm  the  need  to  study  cockpit  performance  along 
multiple  dimensions.  No  single  set  of  indicators  or  single  assessment 
methodology  should  be  considered  sufficient  for  evaluating  cockpit 
management  and  crew  behavior. 


CHAPTER  VI 


CONCLUSIONS  AND  RECOMMENDATIONS 

This  final  chapter  summarizes  the  entire  research  effort.  Con¬ 
clusions  emanating  from  the  accomplishment  of  stated  objectives  are 
presented  first.  Then  comments  based  on  the  examination  of  crew 
coordination  procedures  are  offered.  Finally,  recommendations  concern¬ 
ing  the  methodology  and  the  emphasis  of  future  procedural  research  are 
given. 

Accomplishment  of  Objectives 

In  a  setting  as  structurally  complex  as  a  commercial  airliner, 
analyzing  human  performance  is  an  imprecise  and  introspective  process. 
Identification  of  meaningful  performance  variables  is  necessarily 
dependent  upon  the  evaluator's  experience  and  intuition.  His  under¬ 
standing  of  system  components,  tasks,  and  procedures  determines  the 
importance  placed  on  particular  aspects  of  human  behavior.  Dr. 

Ruffell  Smith  converged  on  the  crews'  interactions  with  control 
mechanisms  and  on  aircraft  parameters  in  relation  to  the  operating 
environment.  This  author  has  focused  on  the  crew  members'  interactions 
with  each  other  in  terms  of  their  affinity  to  published  procedures  and 
management  precepts. 

The  objective  of  defining  a  countably  finite  set  of  Normal  Operat¬ 
ing  Procedures  was  satisfied  in  Chapter  III.  The  SOPs  listed  there 
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represent  highly  distilled  information  from  manifold  sources.  No 
single  document  puts  together  all  of  the  aircraft  subsystems,  airline 
operations,  traffic  control,  navigation,  airport,  communication,  and 
assorted  miscellaneous  procedures  which  are  applicable  to  the  enormous 
range  of  "normal"  operating  conditions.  When  "alternate,"  "abnormal," 
"irregular,"  and  "emergency"  procedures  are  added  to  the  list,  the 
stack  of  crew  directives  becomes  truly  imposing.  Crew  members  face  an 
impossible  challenge  in  attempting  to  mentally  catalogue  all  of  the 
SOPs  published  for  them. 

The  diversified  classifications  or  taxonomies  of  the  Normal  Oper¬ 
ating  Procedures  further  illustrated  the  potential  for  perplexity  in 
temporally  organizing  SOPs.  Crew  directives  seem  to  favor  the  mission 
phase  taxa  suggested  earlier,  but  not  all  publications  employ  them, 
and  none  uses  them  exclusively.  Too  many  procedures  belong  to  more  than 
one  meaningful  taxon.  Aircraft,  company,  and  FAA  procedures  are  often 
presented  sequentially  as  they  might  be  applied  in  flight.  However, 
the  level  of  abstraction  is  inconsistent;  formats  are  diverse;  oper¬ 
ator  involvement  is  not  always  well  defined;  and  integration  of  the 
total  procedural  load  clearly  depends  on  the  intellectual  capacity  and 
operational  experience  of  individual  operators.  Based  on  experimental 
observations  the  extrinsic  value  of  any  particular  routine  procedure 
depended  more  on  crew  composition  and  the  personal  experience  of  the 
users  than  on  any  outsider's  measure  of  its  worth. 

The  objective  of  quantifying  compliance  with  an  observable  sample 
of  the  Normal  Operating  Procedures  was  realized  through  definition  of 
the  Crew  Coordination  category.  That  classification  was  based  on 
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explicit  procedural  imperatives.  The  set  contained  SOPs  which  mandate 
a  verbal  transfer  of  information  from  one  crew  member  to  another. 
Behavioral  equivalence  classes  (checklists,  callouts,  configuration 
changes,  and  transfers)  were  identified  within  the  general  category. 
Multiple  opportunities  for  compliant  behavior  in  each  class  permitted 
counting  of  conformable  activities.  The  individual  operators  and  crews 
in  the  experiment  varied  greatly  in  their  adherence  to  Crew  Coordina¬ 
tion  Procedures.  Since  their  compliance  with  other  SOPs  (e.g.,  flight 
control,  navigation,  and  systems  operation)  varied  as  well,  it  became 
apparent  that  in  any  given  situation  standardized  procedures  are  no 
guarantee  of  uniform  behavior.  Routine  noncompliance  with  an  assort¬ 
ment  of  SOPs  has  now  been  documented  by  two  independent  researchers. 

The  subjective  assessments  of  each  captain's  practice  of  the 
management  precepts  of  continuity,  cooperation,  and  discipline  were  an 
especially  troublesome  aspect  of  this  research.  Since  the  evaluations 
were  made  with  prior  knowledge  of  operator  error  counts  and  crew 
coordination  figures,  a  potential  bias  cannot  be  discounted.  However, 
each  pilot-in-command  was  viewed  in  his  preeminent  role  as  a  resource 
manager.  The  resources  were  personnel,  material,  time,  and  informa¬ 
tion.  The  subjective  measures  reflected  this  author's  confidence  in 
and  agreement  with  each  captain's  management  of  his  resources.  Similar 
judgments  by  subordinates,  peers,  and  other  analysts  would  have  signif¬ 
icantly  increased  the  reliability  of  the  subjective  managerial  assess¬ 
ment.  However,  the  logistics  involved  in  obtaining  such  judgments, 
including  the  nonavailability  of  qualified  crew  members  and  the 
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extensive  time  required,  precluded  their  incorporation  in  this  investi¬ 
gation. 

Another  goal  of  this  research  was  to  attribute  the  "crew"  errors 
identified  in  Dr.  Ruffell  Smith's  original  study  to  specific  operators. 
Analysis  has  revealed  that  many  errors  were  in  fact  committed  by  two  or 
more  crew  members.  Of  the  77  errors  listed  in  Appendix  D,  23  can  be 
ascribed  to  the  pilots  jointly  (PTM)  and  12  others  are  attributed  to 
all  three  crew  members  (CRH).  The  rest  are  distributed  as  follows: 
Captains  (CAP)  20,  copilots  (COP)  20,  and  flight  engineers  (FE)  2. 

This  means  that  for  45%  of  the  enumerated  errors  two  or  more  operators 
were  deficient  simultaneously.  The  message  of  this  statistic  is  that 
human  redundancy  by  itself  does  not  eradicate  personnel  error.  Oper¬ 
ators  must  be  motivated  to  both  give  and  receive  admonitions  concerning 
every  facet  of  cockpit  activity.  Based  on  experimental  observations, 
few  crew  members  appear  to  be  so  motivated. 

The  comparisons  of  objective  measures  of  procedural  compliance 
and  operator  error  with  the  aforementioned  subjective  assessments  of 
managerial  skill  disclosed  that  correspondence  was  greatest  among 
individuals  and  crews  with  the  least  desirable  evaluations.  In  par¬ 
ticular,  the  crew  with  the  most  pilot  errors  (PLTS  =  26  for  run  number 
six)  also  had  the  worst  composite  record  of  compliance  with  crew 
coordination  procedures,  and  the  captain  was  rated  low  on  every 
management  precept.  That  one  crew's  performance  was  so  extreme  that 
it  might  be  considered  an  experimental  anomaly.  However,  the  very 
occurrence  of  such  unacceptable  behavior  by  a  fully  certified  crew 
raises  doubts  about  the  professionalism  and  the  homogeneity  of  the 
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entire  aircrew  population.  In  addition,  the  diverse  assessments  of 
deficiency  on  experimental  runs  twelve  and  thirteen  strongly  reinforced 
the  notion  that  procedural  compliance  and  the  captain's  management 
effectiveness  both  suffer  when  the  copilot  is  flying  a  mission  segment. 
Because  of  the  importance  of  this  finding  and  the  smallness  of  the 
relevant  data  sample,  further  investigation  regarding  this  phenomenon 
is  essential. 

Previously  unrealized  possibilities  for  studying  aircrew  behavior 
in  a  controlled,  high  fidelity,  operational  setting  were  unveiled  in 
Dr.  Ruff ell  Smith's  pioneering  attempt  at  full  mission  simulation.  The 
methodology  permits  simultaneous  observation  of  numerous  details  of 
human  performance  across  a  broad  range  of  reproducible  conditions,  but 
it  is  not  without  drawbacks.  Pull  mission  simulation  is  expensive  and 
time  consuming  to  perform.  It  requires  extensive  preparation  and  can 
yield  overwhelming  amounts  of  data.  Since  individual  air  carriers  own 
most  of  the  sophisticated  simulators,  accessibility  is  often  limited  by 
high  utilization  rates  for  crew  training  and  certification.  Neverthe¬ 
less,  simulator  data  represents  the  best  approximation  to  operational 
data  attainable  today. 

Routine  flight  operations  have  never  been  open  to  impartial 
research.  The  FAA,  the  airlines,  and  the  pilots'  union  have  been 
reluctant  to  permit  public  inspection  of  their  domain.  Except  for 
full  mission  simulation  data,  carefully  edited  and  highly  distilled 
accident  reports  from  the  NTSB  provide  the  only  available  documentation 
of  cockpit  behavior  which  addresses  all  the  complexities  of  the  oper¬ 


ating  environment 
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Future  attempts  at  full  mission  simulation  should  utilize  lessons 
contained  in  the  airline  experiment  and  in  a  more  recent  general  avi¬ 
ation  experiment  at  this  university.  To  begin  with,  video  data  should 
be  incorporated  to  disclose  control  movements,  false  starts,  attention 
patterns,  visual  signals,  and  physical  excitability.  To  the  extent 
practical,  routes,  airports,  and  mission  sequences  familiar  to  the 
participating  crews  should  be  used  to  establish  base  line  data.  Pro¬ 
fessional  Air  Traffic  Controllers  should  be  involved  in  the  development 
of  simulation  support  materials  like  background  audio  tapes  and,  if 
possible,  in  scenario  execution,  performing  their  usual  functions. 
Finally,  the  experimental  population  should  represent  a  cross  section 
of  the  total  crew  force.  Participations  should  not  be  restricted  to 
volunteers.  Crew  members  should  be  scheduled  for  full  mission  simula¬ 
tions  in  the  same  way  that  they  are  scheduled  for  routine  flights 
(seniority  bidding) .  Recreation  of  operational  detail  is  its  strong¬ 
est  selling  point  with  crew  members. 

Crew  Coordination  Procedures 

The  verbal  crew  coordination  imperatives  identified  in  this 
research  are  a  small  fraction  of  any  crew  member's  procedural  inven¬ 
tory.  However,  they  can  be  just  as  crucial  to  flight  safety  as  any 
other  category  of  SOPs.  Failure  to  comply  with  routine  callout  pro¬ 
cedures  directly  contributed  to  the  BAC  1-11  accident  cited  in 
Chapter  II  (34).  Failure  to  properly  transfer  control  has  been 
associated  with  an  infamous  crash  in  the  Everglades  (37). 
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The  experimental  data  demonstrated  that  noncompliance  with  inter¬ 
nal  coordination  requirements  was  not  restricted  to  any  single  SOP, 
crew  position,  level  of  experience,  or  phase  of  operations.  Every 
crew  violated  procedures,  but  no  deviations  were  common  to  all  crews. 
Nonconforming  behavior  appeared  to  be  uniquely  operator  and/or  crew 
dependent.  The  behavioral  enumerations  given  in  Chapter  IV  illustrate 
a  breadth  and  depth  of  noncompliance  which  must  be  assumed  to  occur 
throughout  daily  airline  operations. 

Among  the  verbal  coordination  procedures  studied,  the  altitude 
callouts  prior  to  level  off  produced  a  frequency  of  noncompliant 
behavior  which  suggested  a  potential  need  for  modification.  Since  the 
mechanical  altitude  alerting  system  provided  visual  as  well  as  audible 
indications  of  proximity  to  a  pre-selected  altitude,  the  prescribed 
verbal  callout  seemed  unnecessary.  From  a  human  factors  standpoint  the 
SOP  mandates  operator  activity  comparable  to  that  already  being  per¬ 
formed  by  a  machine.  Since  machines  accomplish  such  tasks  more 
reliably  than  humans,  the  latter  should  either  have  their  procedural 
load  reduced  or  be  given  a  different  task  which  does  not  mimic  a 
machine  function. 

The  most  disturbing  aspect  of  the  experimental  data  concerned  the 
lack  of  unitary  leadership  and  internal  coordination  observed  when  the 
captain  was  not  flying  the  aircraft.  Normal  operating  procedures  of 
the  participating  airline  appear  to  have  been  designed  specifically  for 
flying  captains.  The  directives  do  not  define  the  roles  of  a  flying 
copilot  and  nonflying  captain  in  sufficient  detail.  A  standardized  pro¬ 
cedure  for  redistribution  of  tasks  and  authority  is  not  provided. 
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A  captain  can  theoretically  delegate  all  of  his  authority  to  the 
copilot,  but  the  final  responsibility  for  safe  and  proper  accomplish¬ 
ment  of  a  flight  mission  can  never  be  delegated.  It  always  belongs  to 
the  pilot-in-command .  Since  most  captains  regularly  alternate  flying 
segments  with  their  copilots,  a  simple  addition  to  existing  procedures 
seems  in  order.  The  pilot-in-command  should  be  required  to  inform  the 
entire  crew  concerning  the  degree  of  functional  authority  he  is  giving 
to  the  copilot.  He  should  also  state  the  degree  to  which  he  himself 
will  perform  the  support  task  of  the  nonflying  pilot .  These  items 
could  be  prescribed  as  part  of  the  initial  briefing  to  cabin  and  cock¬ 
pit  crew  members. 

Although  this  research  has  produced  only  two  recommended  changes 
to  existing  SOPs  (for  altitude  callouts  and  flying  copilot  responsibil¬ 
ities),  it  has  highlighted  several  facets  of  the  relationship  between 
operational  safety  and  aircrew  procedures.  Most  importantly,  simulator 
data  and  NTSB  accident  reports  have  shown  that  SOPs  are  not  a  panacea 
for  mechanical,  environmental,  or  personnel  problems.  Even  if  pro¬ 
cedures  are  available  for  every  anticipated  situation,  they  must  be 
learned,  practiced,  recalled,  and  applied  by  fallible  human  operators. 

When  those  operators  create  internal  models,  SOPs  are  a  portion  of  the 
input;  but  an  unknown  amount  of  self-procedurization  goes  on  continuously. 

Future  Research 

Future  investigations  of  procedural  compliance  and  the  role  of 
SOPs  in  aviation  safety  could  proceed  using  several  different  methodolo¬ 
gies.  In  this  researcher's  opinion  full  mission  simulation  is  the  most 


121 


promising;  however,  it  also  consumes  the  most  resources.  For  that 
reason  alone  other  methodologies  must  be  entertained. 

Data  from  NASA's  confidential  Aviation  Safety  Reporting  System 
could  be  surveyed  to  establish  the  kinds  of  procedural  noncompliance 
which  cause  some  crew  members  to  acknowledge  a  dangerous  experience. 
Aircrew  interviews  could  elicit  recollections  of  critical  incidents 
resulting  from  nonadherence  to  SOPs.  Formal  surveys  of  crew  members 
could  be  used  to  establish  frequency  and  criticality  indices  for 
specific  types  or  circumstances  of  noncompliance.  Lower  fidelity  simu¬ 
lations  can  include  procedural  knowledge  testing  in  paper  and  pencil 
senarios  at  a  desk,  interactive  computer  scenarios  at  a  video  terminal, 
task  specific  activity  in  an  aircraft  simulator,  or  any  other  imita¬ 
tion  of  flight  conditions.  The  relative  merits  of  these  diverse 
methodologies  have  not  been  substantiated  in  comparative  research,  but 
for  this  author  the  emotional  realism  of  full  mission  simulation  makes 
its  appeal  overwhelming. 

Since  the  current  number  of  data  points  on  procedural  compliance 
is  small,  large  scale  experiments  involving  participation  by  distinc¬ 
tive  groups  of  operators  should  be  attempted.  For  example,  the  crews 
of  different  air  carriers  should  be  exposed  to  a  common  experimental 
scenario.  Differences  in  the  applicable  procedures  or  levels  of  com¬ 
pliance  could  lead  to  significant  insights  regarding  the  effectiveness 
of  SOP  formats  and  content  as  well  as  operator  training  and  motivation. 

Ideally,  one  or  two  full  mission  simulation  scenarios,  complete 
with  data  gathering  guidelines,  should  be  included  in  the  recurrent 
training  program  of  every  airline.  The  next  generation  of  aircraft 


simulators  should  accommodate  audio,  video,  and  flight  parameter  data 
collection  at  any  time.  Crew  members,  instructors,  flight  examiners, 
and  analysts  from  numerous  disciplines  could  then  review  aircrew  per¬ 
formance  along  several  dimensions. 

The  safety  record  compiled  by  the  commercial  aviation  industry 
in  this  country  is  truly  commendable,  but  there  is  reason  to  believe  it 
can  be  improved.  The  National  Transportation  Safety  Board  has  con¬ 
sistently  pointed  to  deviations  from  explicit  operating  procedures  as 
causal  and  contributory  factors  in  airline  accidents.  Until  airline 
cockpits  are  opened  to  operational  research,  full  mission  simulation 
represents  the  most  sensitive  medium  for  evoking  and  detecting  the 
behavioral  nuances  which  can  turn  ordinary  noncompliance  into  an 
extraordinary  aviation  tragedy. 


APPENDIX  A 


ABBREVIATIONS  AND  ACRONYMS 

AAR  Aircraft  Accident  Report 

ADF  Automatic  direction  finder  (a  navigation  radio) 

ADH  Above  decision  height 

AFE  Above  field  elevation 

AGL  Above  ground  level 

ALPA  Air  Line  Pilots  Association 

Alt.  Altitude 

AOM  Aircraft  Operating  Manual 

ALT  HOLD  the  Altitude  Holding  mode  of  the  autopilot 

A/P  Autopilot 

App.  Approach 

ARTCC  Air  Route  Traffic  Control  Center 

A/S  airspeed 

ATA  Air  Transport  Association 

ATC  Air  Traffic  Control 

ATIS  Automatic  Terminal  Information  Service 

Att  attack,  as  in  aircraft  angle  of  attack 

BAC  British  Aircraft  Corporation 

CAP  count  of  errors  committed  by  the  captain  alone 

CCAP  sum  of  errors  committed  by  the  captain  alone,  by  the 

pilot  team,  and  by  the  entire  crev 
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CCOP 

sum  of  errors  conmitted  by  the  copilot  alone,  by  the  pilot 
team,  and  by  the  entire  crew 

CMD 

The  Command  mode  of  the  autopilot 

COM 

Company  Operations  Manual 

comm. 

communications 

COP 

count  of  errors  committed  by  the  copilot  alone 

CPF 

sum  of  errors  committed  by  the  flying  pilot  alone,  by 
pilot  team,  and  by  the  entire  crew 

the 

CPNF 

sum  of  errors  committed  by  the  nonflying  pilot  alone, 
pilot  team  and  by  the  entire  crew 

by  the 

CREX 

crew  experience  variable  (discrete) 

CRW 

count  of  errors  committed  jointly  by  the  entire  crew 

CVR 

Cockpit  Voice  Recorder 

dec. 

decision 

ECG 

electrocardiograph 

EGT 

exhaust  gas  temperature 

EPR 

engine  pressure  ratio 

FAA 

Federal  Aviation  Administration 

FAF 

final  approach  fix 

FAR 

Federal  Aviation  Regulation 

F/D 

flight  director 

FDR 

Flight  Data  Recorder 

FE 

flight  engineer 

FEA 

count  of  flight  engineer  checklist  announcements 

FF 

fuel  flow 

flaps(  )  wing  flaps  (angle  of  extension) 
FM  frequency  modulation 
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GPWS 

IAS 

ILS 

INS 

JFK 

(  )K 

MAN 

MDA 

MSL 

NASA 

nav 

NOTAM 

NTSB 

N1 

OM 

PA 

PAT  CO 

PEX 

PF 

PFCK 

PLTS 

PNF 

PNFC 

PTM 

PI 

P2 


GroiinJ  Proximity  Warning  System 
indicated  air  speed 
Instrument  Landing  System 
Inertial  Naviation  System 

John  F.  Kennedy  International  Airport  (New  York,  New  York) 

(  )  knots  [airspeed] 

the  Manual  mode  of  the  autopilot 

minimum  descent  altitude 

mean  sea  level 

National  Aeronautics  and  Space  Administration 

navigation 

notice  to  airmen 

National  Transportation  Safety  Board 
first  stage  compressor  of  a  turbofan  engine 

outer  marker 

count  of  pilot  checklist  announcements 
Professional  Air  Traffic  Controllers  Organization 
pilot  team  experience  variable  (discrete) 

pilot  flying;  count  of  errors  committed  by  the  flying  pilot 
alone 

count  of  checklist  commands  by  the  flying  pilot 

sum  of  errors  by  the  two  pilots  (PF  +  PNF  +  2(PTM)  +  2(CRW)) 

pilot  not  flying 

count  of  callouts  by  the  pilot  not  flying 

count  of  errors  committed  by  the  two  pilots  jointly 

captain 

copilot 


i 
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SOP 

STAR 

TAS 

TCAP 

TCOP 

T/O 

TPF 

TPNF 

TRAN 

TURB 

VOR 


standard  operating  procedure 
Standard  Terminal  Arrival  Route 
true  air  speed 

sum  of  errors  committed  by  the  captain  alone  and  by  the 
pilot  team 

sum  of  errors  committed  by  the  copilot  alone  and  by  the 
pilot  team 

takeoff 

sum  of  errors  committed  by  the  flying  pilot  alone  and  by 
the  pilot  team 

sum  of  errors  committed  by  the  nonflying  pilot  alone  and 
by  the  pilot  team 

count  of  aircraft  control  transfers 

the  Turbulence  mode  of  the  autopilot 

very  high  frequency  omnirange  (a  navigation  radio) 


V. 
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ANALYSIS  DIAGRAM 
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APPENDIX  C 

EXAMPLES  OF  PICTORIAL  AND 
GRAPHICAL  PROCEDURES 
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TABLE  26 


ERROR  COUNTS 


Run 

No. 


Ruffell  Smith 
Categories 


_ By-Operator  Categori 

No.  PF  CAP  PNF  COP  PTM 


3  Communication 
Flying 

Tactical  dec. 
Flying  skill 
Autopilot 
Total 


1 

2 

1 

2 

1 

7 


1  1 

2  2 

1 
1 


1  1 

1  1 

4  4  11 


4  Tactical  dec. 
Flying  skill 
Autopilot 
Other 
Total 


1 

1 

1 

1 

4 


1  1 

1  1 

1111 


1 

1 

2 


5  Navigation 

System  operation 

Tactical  dec. 

Autopilot 

Other 

Total 


1 

1 

3 

1 

1 

7 


1 

1 

1  1 

110  0  2 


6  Navigation  1 

Communication  5 

System  operation  6 

Tactical  dec.  1 

Crew  integration  1 

Flying  skill  3 

Autopilot  2 

Total  19 


1 

4  4  1 

2  112  1 

1  1 

3  3 

1  11 

5  5  5  5  4 


8  Navigation  3 

Communication  1 

System  operation  1 

Flying  2 

Flying  skill  1 

Total  8 

10  Communication*3  1 

Navigation  1 

System  opeartion  1 

Flyingc  1 

Tactical  dec.  1 

Autopilot  1 

Total  4 


11111 
1  1 

2 

1  1 

2  2  2  2  3 

1  1 

1  1 

1 


1 

0  0  112 


TABLE  26  (Contd.) 
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Run 

No. 

Ruf fell  Smith 
Categories 

No. 

By-Operator  Categories 

PF 

CAP 

PNF 

COP 

PTM 

FE 

CRW 

12 

Communicat ion^ 

1 

1 

1 

Navigation 

2 

2 

2 

Flying 

3 

1 

1 

2 

Tactical  dec. 

1 

1 

Total 

7 

1 

3 

3 

1 

2 

0 

1 

13. 

Navigation6 

1 

Communication 

4 

1 

1 

3 

System  operation 

1 

1 

Flying 

1 

1 

Tactical 

1 

1 

1 

Flying  skill 

2 

1 

1 

1 

1 

Autopilot 

2 

2 

2 

Total 

11 

4 

2 

2 

4 

4 

0 

1 

14 

Navigation 

1 

1 

1 

Communication 

3 

3 

3 

Autopilot 

1 

1 

Other 

1 

1 

Total 

6 

0 

0 

4 

4 

1 

0 

1 

15. 

Flying 

2 

2 

Tactical  dec. 

1 

1 

Autopilot 

1 

1 

1 

Total 

4 

1 

1 

0 

0 

2 

0 

1 

aError  found  during  crew  coordination  procedures  analysis. 
^Error  found  during  crew  coordination  procedures  analysis. 

Error  could  not  be  verified  by  the  data. 

^Error  found  during  crew  coordination  procedures  analysis. 
eSecond  segment  error  improperly  listed  for  first  segment. 
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TABLE  27 

ERROR  DESCRIPTIONS 

Run 

Ruffell  Smith 

By-Operator 

No. 

Cat  egories 

Descript  ion 

Categories 

3 

Communication3 

P2  called  wrong  ATC  station 

PNF  COP 

Flying 

PI  follows  F/D  blindly 

PF  CAP 

Flying 

flap  25  too  early 

PTM 

Tactical  dec. 

flap  25  landing 

CRW 

Flying  skill 

PI  throttle  control 

PF  CAP 

Flying  skill 

alt.  excursion  near  end  of  cruise 

PTM 

Autopilot 

disengagement  caused  by  PI 

PF  CAP 

4 

Tactical  dec. 

raise  gear  before  800’  AFE 

PTM 

Flying  skill 

P2  rough 

PNF  COP 

Autopilot 

TURB  mode  not  selected 

PTM 

Other 

PI  short  term  memory 

PF  CAP 

5 

Navigation 

nav  switches  in  ADF  for  T/0 

PTM 

Sys.  operation 

P3  drains  reserve  fuel  tank 

FE 

Tactical  dec. 

no  ice  protection  in  thunderstorm 

CRW 

Tactical  dec. 

flap  5  at  800'  AFE  on  T/0 

PTM 

Tactical  dec. 

flap  25  landing 

CRW 

Autopilot 

PI  never  tries  to  use 

PF  CAP 

Other 

P3  uses  only  1  side  of  shld.  harness 

FE 

6 

Navigation 

nav  switches  in  ADF  for  T/0 

PTM 

Communication 

PI  fails  to  contact  ATC  (tower) 

CAP  PNF 

Communication 

PI  copies  wrong  squawk 

CAP  PNF 

Communication 

PI  misses  T/0  clnc 

CAP  PNF 

Communication 

PI  uses  wrong  call  sign 

CAP  PNF 

Communication 

does  not  report  level  at  4000' 

PTM 

Sys.  operation 

P2  puts  flaps  down  during  start 

PF  COP 

Sys.  operation 

no  ice  protection  on  T/0 

CRW 

Sys.  operation 

P3  reduces  power  as  EPR  increase 

CRW 

Sys.  operation 

P2  selects  wrong  computer  for  F/D 

PTM 

Sys.  operation 

P2  resets  B  computer  &  loses  F/D 

PF  COP 

Sys.  operation 

Pi  difficulty  tuning  ADFs 

CAP  PNF 

Tactical  dec. 

PI  fails  to  take  over  unstable  app 

CAP  PNF 

Crew  integration 

no  comm,  regarding  setting  radios 

CRW 

Flying  skill 

P2  follows  F/D  too  late 

PF  COP 

Flying  skill 

P2  excessive  angle  at  att.  in  turn 

PF  COP 

Flying  skill 

P2  throttle  control  on  final 

PF  COP 

Autopilot 

P2  put  A/P  to  CMD  w/o  pause  at  MAN 

PF  COP 

Autopilot 

in  CMD  during  turbulence 

PTM 
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TABLE  27  (Contd.) 


Run 

No. 

Ruffell  Smith 
Categories 

Description 

By-Operator 

Categories 

8 

Navigation 

P2  tunes  wrong  OM  at  JFK 

PNF  COP 

Navigation 

ILS  mode  not  selected 

PTM 

Navigation 

PI  misreads  navaid  frequency 

PF  CAP 

Communicat ion 

P2  copies  wrong  frequency 

PNF  COP 

Sys.  operation 

no  ice  protection  on  T/0 

CRW 

Flying 

low  A/S  and  high  rate  of  descent 

on  final 

PTM 

Flying 

270K  at  5000'  in  climb 

PTM 

Flying  skill 

PI  throttle  control  on  final 

PF  CAP 

10 

Navigation  . 

P2  sets  ADF  incorrectly 

PNF  COP 

Communication 

P2  uses  wrong  call  sign 

PNF  COP 

Sys.  operation 

P2  selects  wrong  computer  for  F/D 

PTM 

Fly ingc 

Tactical  dec. 

flap  25  landing 

CRW 

Autopilot 

TURB  mode  not  selected 

PTM 

12 

Navigation 

PI  problems  w/  radials  after  T/0 

CAP  PNF 

Navigation 

PI  does  not  understand  way  pt  chg 

CAP  PNF 

Communication 

PI  talks  to  cabin  instead  of  ATC 

CAP  PNF 

Flying 

over  rotation  on  takeoff 

PTM 

Flying 

neither  pilot  has  control 

PTM 

Flying 

excessive  hand  flying  in  climb 

PF  COP 

Tactical  dec. 

flap  25  landing 

CRW 

13 

e 

Navigation 

PTM 

Communication 

PI  does  not  call  tower  after  FAF 

PTM 

Communication 

PI  calls  NY  Center  on  wrong  freq. 

CAP  PNF 

Communication 

PI  does  not  report  Southgate 

PTM 

Communicat ion 

does  not  squawk  when  requested 

PTM 

Sys.  operation 

P3  sets  //lEPR  higher  than  others 

CRW 

Flying 

flap  5  too  early 

PTM 

Tactical  dec. 

P2  wants  to  make  flap  25  landing 

PF  COP 

Flying  skill 

PI  rough 

CAP  PNF 

Flying  skill 

P2  throttle  control  on  approach 

PF  COP 

Autopilot 

not  used  in  climb 

PF  COP 

Autopilot 

P2  puts  to  CMD  while  in  ALT  HOLD 

PF  COP 

14 

Navigat ion 

P2  sets  wrong  VOR  frequency 

PNF  COP 

Communicat ion 

P2  does  not  use  call  sign 

PNF  COP 

Communicat ion 

P2  uses  wrong  call  sign 

PNF  COP 

Communicat ion 

P2  asks  for  repeat  of  heading 

PNF  COP 

Autopilot 

TURB  mode  not  used 

PTM 

Other 

altimeter  errors  not  noticed 

CRW 
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TABLE  27  (Contd.) 


Run 

Ruff ell  Smith 

By-Operator 

No. 

Categories 

Description 

Categories 

15 

Flying 

326K  at  9000’ 

PTH 

Flying 

flaps  20  too  early 

PTM 

Tactical  dec. 

Descent  Check  started  late 

CRW 

Autopilot 

PI  uses  wrong  A/P 

PF  CAP 

aError  found  during  crew  coordination  procedures  analysis. 
bError  found  during  crew  coordination  procedures  analysis. 
CError  could  not  be  verified  by  the  data. 

^Error  found  during  crew  coordination  procedures  analysis. 
eSecond  segment  error  improperly  listed  for  first  segment. 


APPENDIX  E 


EVALUATION  GUIDES 


Crew  Coordination 

In  the  main  the  Crew  Coordination  Procedures  occur  in  a  sequence 
determined  by  the  experimental  scenario.  However,  they  are  inter¬ 
twined  with  numerous  other  verbal  communicat ions  including  ATC  clear¬ 
ances,  ATIS  recordings,  messages  to  or  from  Operations,  passenger 
announcements,  and  general  conversation.  Crew  members  frequently 
interrupt  or  preempt  one  another.  When  prescribed  verbalizations  are 
seemingly  omitted,  extra  vigilence  must  be  maintained  to  guarantee  that 
a  required  statement  is  not  masked  by  other  communications.  Also, 
proceduralized  information  may  be  literally  correct  but  may  occur  at 
the  wrong  time  and/or  in  the  wrong  sequence.  Hence,  in  recording 
occurrences  of  compliance  with  Crew  Coordination  Procedures  it  is 
essential  to  note  specific  declarations,  the  spokesman,  and  the  se¬ 
quence  of  operational  events.  Table  28  shows  procedure  names,  scored 
compliance  activity,  and  appropriate  operators  in  an  idealized 
sequence.  The  Transfer  of  Aircraft  Control  procedure  is  timely  when¬ 
ever  it  does  not  interfere  with  the  accomplishment  of  other  tasks. 
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TABLE  28 

CREW  COORDINATION  COMPLIANCE  SCORING 


Procedure  Name 

Prescribed  Verbalizations 

Prescribed 

Operators 

Pre-Start  Checklist 

Command 

PF 

Announcement 

PNF 

Challenges  and  responses 

PNF, 

PF, 

FE 

Completion  statement 

PNF 

Start  Checklist 

Command 

PF 

Announcement 

PNF 

Challenges  and  responses 

PNF, 

PF, 

FE 

Completion  statement 

PNF 

Pre-Taxi  Checklist 

Command 

PF 

Announc  emen  t 

PNF 

Challenges  and  responses 

PNF, 

PF 

Completion  statement 

PNF 

Transfer  of  EGT  Monitor 

Acceptance 

FE 

Relinquishment 

PF 

Taxi  Checklist 

Command 

PF 

Ammouncement 

PNF 

Challenges  and  responses 

PNF, 

PF, 

FE 

Completion  statement 

PNF 

Takeoff  Checklist 

Command 

PF 

Announcement 

PNF 

Challenges  and  responses 

PNF, 

PF, 

FE 

Completion  statement 

FE 

Takeoff  Callouts 

Airspeed 

PNF 

80  knots 

PNF 

VI  [safe  decision  speed] 

PNF 

VR  [rotation  speed] 

PNF 

V2 

PNF 

Positive  rate  [of  climb] 

PNF 

Gear  Retraction 

Command 

PF 

Acknowledgement 

PNF 

Takeoff  Callout 

800  feet 

PNF 
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TABLE  28  (Contd.) 


Procedure  Name 

Prescribed  Verbalizations 

Prescribed 

Operators 

Flap  Retraction 

Command  [flaps  5] 

PF 

Acknowl edgement 

PNF 

Command  [flaps  1] 

PF 

Acknowledgement 

PNF 

Command  [flaps  up] 

PF 

Acknowledgement 

PNF 

Altitude  Callout 

1000  to  go  [at  3000' 

MSL] 

PNF 

After  Takeoff  Checklist 

Command 

PF 

Completion  statement 

FE 

Altitude  Callout 

1000  to  go  [at  14000' 

1  MSL] 

PNF 

Transfer  of  Aircraft 

Acceptance 

PNF 

Control 

Relinquishment 

PF 

Acceptance 

PF 

Relinguishment 

PNF 

Descent  Checklist 

Command 

PF 

Announcement 

FE 

Challenges  and  responses 

FE, 

PF,  PNF 

Completion  statement 

FE 

Approach  Checklist 

Command 

PF 

Announcement 

FE 

Challenges  and  responses 

FE, 

PF,  PNF 

Completion  statement 

FE 

Altitude  Callout 

1000  to  go  [at  5000' 

MSL] 

PNF 

Approach  Flap  Extension 

Command  [flaps  1] 

PF 

Acknowledgement 

PNF 

Command  [flaps  5] 

PF 

Acknowledgement 

PNF 

Command  [flaps  10] 

PF 

Acknowledgement 

PNF 

Command  [flaps  20] 

PF 

Acknowledgement 

PNF 
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TABLE  28  (Contd.) 


Procedure  Name 

Prescribed  Verbalizations 

Prescribed 

Operators 

Landing  Gear/Landing 

Command  [gear  down] 

PF 

Flap  Extension 

Acknowl edgement 

PNF 

Command  [flaps  25] 

PF 

Acknowledgement 

PNF 

Command  [flaps  30] 

PF 

Acknowledgement 

PNF 

Landing  Checklist 

Command 

PF 

Announcement 

FE 

Challenges  and  responses 

FE,  PF 

Completion  statements 

FE 

Precision  Approach 

Outer  marker 

PNF 

Callouts 

500  feet  AFE 

PNF 

200  feet  ADH 

PNF 

100  feet  ADH 

PNF 

Minimums 

PNF 

Landing  Roll  Callouts 

All  in  reverse 

FE 

100  knots 

PNF 

90  knots 

PNF 

60  knots 

PNF 

All  out  of  reverse 

FE 

For  each  of  the  above  procedures  an  instance  of  noncompliance  is 
recorded  whenever  the  required  verbalization  is  omitted,  uttered  out 
of  sequence,  or  spoken  by  other  than  the  prescribed  operator.  In  the 
case  of  callout  procedures,  statements  which  do  not  concur  with  actual 
aircraft  parameters  (e.g.,  an  early  or  late  declaration  of  "1000  feet 
to  level  off")  are  also  counted  as  cases  of  noncompliance. 


Cockpit  Management 

In  every  type  of  aviation  pilots  are  obliged  to  undergo  recur¬ 
rent  subjective  evaluations  of  their  flight  management  skills.  For 
general  aviation  pilots,  a  biennial  flight  review  from  an  FAA  examiner 


man  imp 
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or  a  certified  flight  instructor  is  required.  For  Air  Force  pilots  a 
minimum  of  two  standardization/ evaluation  flights  per  year  is  normally 
mandatory;  in  most  military  units  periodic  evaluations  in  a  simulator 
are  also  required.  Airline  pilots  receive  semi-annual  flight  checks 
and  simulator  evaluations  from  FAA  inspectors  or  designated  company 
examiners . 

Although  specific  tasks  and  procedures  may  vary  greatly  from  one 
aircraft  to  another  and  from  one  operating  environment  to  another 
(e.g.,  single  pilot  general  aviation  flying,  military  bombardment,  and 
transcontinental  passenger  flight) ,  all  pilot  evaluations  involve  one 
aviator's  subjective  assessment  of  another's  composite  airmanship. 

During  every  rating  the  management  principles  of  continuity  and 
discipline  are  judged,  sometimes  implicitly  but  often  explicitly. 
Likewise,  in  any  cockpit  occupied  by  more  than  one  crew  member  the 
precept  of  cooperation  can  hardly  be  overlooked.  The  need  for  cooper¬ 
ation  is  perceptible  in  every  multi-place  aircraft.  In  short,  the  type 
of  subjective  assessments  of  cockpit  management  which  are  shown  in 
Chapter  V  are  made  hundreds  of  times  each  day  throughout  the  whole 
spectrum  of  aviation. 

This  author's  faculty  for  judging  other  pilots’  management  skills 
rests  upon  seventeen  years  of  affiliation  with  the  United  States  Air 
Force,  including  four  years  of  formal  education  and  over  twelve  years 
of  piloting.  Flight  training  and  operational  experience  have  been 
accumulated  in  six  different  aircraft  flying  with  sundry  crew  sizes  under 
a  variety  of  environmental  conditions.  The  judgments  contained  herein 
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could  reasonably  have  been  made  by  any  aviator  with  similar  training 
and  experience.  However,  just  as  no  two  flight  examiners  have  exactly 
the  same  perceptions  and  biases,  no  two  surrogate  evaluators  of  the 
experimental  data  should  be  expected  to  produce  identical  assessments. 
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Figure  6.  Normal  Probability  Plot  of  Test  Statistic  for  CPF 
Distribution. 
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Figure  8. 
Distribution. 


Normal  Probability  Plot  of  Test 


Statistic  for  PLTS 


99  *  99.9 


J 


APPENDIX  G 

SUBJECTIVE  ASSESSMENT  DIAGRAMS 


151 


i 


CAP 

Errors 


*  12 
«  13 


v  8 

x  5 

x  10 


M 

Continuity 


COP 

Errors 


6  -  x  6 
5  - 

4  -  x  13 


3  - 
2  - 

1 

0  L 


x  12 


j— 

L 


x  8,10 
x  3 
*5 


M 

Continuity 


Figure  9.  Continuity  Plot 


PTM 

Errors 


CRW 

Errors 


Figure  9.  (Contd.) 


CREX 


«  4,14,15 


6 


5 

4 

TRAN 

3 

2 

1 

0 


Hours 
as  B-747 
captain 

5000 


4000 


3000 

2000 


1000 


CAP 

Errors 


COP 

Errors 


Figure  10.  Cooperation  Plots 


PTM 

Errors 


2 


*  12 


x  4,10 


I 


CRW 

Errors 


Fig.  10.  (Contd.) 


3 


CREX 


PEX 


Cooperation 
Fie.  10.  fContd.l 


Cooperation 


Hours 
as  B-747 
captain 


Figure  10.  (Contd.) 


6 


CAP 

Errors 


COP 

Errors 


Figure  11.  Discipline  Plots 


163 


PTM 

Errors 


CRW 
Errors 


Figure  11.  (Contd.) 


166 


BIBLIOGRAPHY 


1.  Aircraft  Owners  and  Pilots  Association,  AOPA  Aviation  Fact  Card, 
Washington,  D.C.:  undated  (data  current  as  of  September,  1979). 

2.  Anonymous.  Herculean  Snowmobile,  Approach,  Vol.  25,  No.  4,  pp .  16-17. 
(October,  1979) 

3.  Barr,  A.  J.,  Goodnight,  J.  H. ,  Sail,  J.  P.,  and  Helwig,  J.  T.  A  User 1 s 
Guide  to  SAS76.  Raleigh,  North  Carolina:  SAS  Institute,  1976. 

4.  Bailey,  K.  Remarks  at  the  Air  Transport  Association's  1977  Airlines 
Operations  Forum  (October  19,  1977),  National  Transportation  Safety 
Board,  Washington,  D.C.  (undated). 

5.  Bruggink,  G.  M.  Managing  Emergencies  in  Jet  Transport  Operations  in 
Resource  Management  on  the  Flight  Deck.  Cooper,  G.  E.,  White,  M.D., 
and  Lauber,  J.  K. ,  Eds.  NASA  CP-2120,  Washington:  March  1980. 

6.  _ _.  Calculated  Risk  or  Blind  Assumption?  in  Resource  Management 

on  the  Flight  Deck.  Cooper,  G.E.,  White,  M.  D. ,  and  Lauber,  J.  K. , 

Eds.  NASA  CP-2120,  Washington:  March  1980. 

7.  Burks,  E.  L.,  Engler,  H. ,  and  Sears,  W.  E.  III.  Human  Operator  Control 
Strategy  Model  in  1979  Review  of  Air  Force  Sponsored  Basic  Research: 
Flight  and  Technical  Training.  Bolling  AFB,  D.C.:  AFOSR,  1979. 

8.  Connor,  T.  M.  and  Hamilton,  C.  W.  Evaluation  of  Safety  Programs  with 
Respect  to  the  Causes  of  Air  Craft  Accidents.  Columbus,  Ohio: 

Battelle  Columbus  Laboratories,  May  16,  1979. 

9.  Cooper,  G.  E. ,  White,  M.  D.,  and  Lauber,  J.  K. ,  Eds.  Resource  Manage¬ 
ment  on  the  Flight  Deck:  Proceedings  of  a  NASA/Industry  Workshop  at 
San  Francisco,  California,  June  26-28,  1979,  NASA  CP-2120,  Washington: 
March  1980. 

10.  Dam os,  D.  Training  Efficient  Multiple-Task  Strategies  in  1979  Review 
of  Air  Force  Sponsored  Basic  Research:  Flight  and  Technical  Training. 
Bolling  AFB,  D.C.:  AFOSR,  1979. 

11.  Damos,  D.  L.  Residual  Attention  as  a  Predictor  of  Pilot  Performance. 
Human  Factors,  Vol.  20,  No.  4,  pp.  435-440  (August  1978). 


167 


168 


12.  Dreyfus,  H.  L.  and  Dreyfus,  S.  E.  Forman  Versus  Situational  Models 
of  Expert  Decision  Making  in  1979  Review  of  Air  Force  Sponsored  Basic 
Research:  Flight  and  Technical  Training.  Bolling  AFB ,  D.C.:  AFOSR, 

1979. 

13.  Fleishman,  E.  A.  Toward  a  Taxonomy  of  Human  Performance.  American 
Psychologist,  Vol.  30,  No.  12,  December  1975,  pp.  1127-1149. 

14.  Forehlich,  D.  V.  Aircrew  Judgment — Not  Gone  Or  Forgotten,  Aerospace 
Safety,  August,  1978. 

15.  Funk,  K.  Personal  communication.  April  1,  1980. 

16.  _  and  Miller,  R.  A.  Preliminary  Development  of  a  Goals/Resources/ 

Tasks  Description  of  Manned  Systems.  Department  of  Industrial  Engi¬ 
neering,  The  Ohio  State  University,  Columbus,  Ohio  (undated). 

17.  Gartner,  W.  B.  and  Murphy,  M.  R.  Pilot  Workload  and  Fatigue:  A  Crit¬ 
ical  Survey  of  Concepts  and  Assessment  Techniques,  NASA  TN  D-8365, 
Washington,  D.C.:  National  Aeronautics  and  Space  Administration, 
November,  1976. 

18.  Glines,  C.  V.  "A  Statesmanlike  Challenge,"  Air  Line  Pilot,  October, 

1978,  pp.  6-13. 

19.  Johannsen,  G.  Workload  and  Workload  Management  in  Mental  Workload: 

Its  Theory  and  Measurement.  Moray,  N.,  Ed.  New  York:  Plenum  Press, 

1979. 

20.  _ ,  Moray,  N. ,  Pew,  R.,  Rasmussen,  J.,  Sanders,  A.,  and  Wickens,  C. 

Final  Report  of  Experimental  Psychology  Group  in  Mental  Workload:  Its 
Theory  and  Measurement.  Moray,  N. ,  Ed.  New  York:  Plenum  Press,  1979. 

21.  Kinkade,  R.  G.  and  Wheaton,  G.  R.  Training  Device  Design  in  Human 
Engineering  Guide  to  Equipment  Design  (Revised  Edition) ,  VanCott  and 
Kinkade,  Eds.  Washington:  U.S.  Government  Printing  Office,  1972, 

22.  Kirchner,  J.-H.  and  Laurig,  W.  The  Human  Operator  in  Air  Traffic 
Control  Systems,  Ergonomics,  Vol.  14,  No.  5,  pp.  549-556  (1971). 

23.  Kowalsky,  N.  B.,  Masters,  R.  L.,  Stone,  R.  B.,  Babcock,  G.  L. ,  and 
Rypka,  E.  W.  An  Analysis  of  Pilot-Error-Related  Aircraft  Accidents. 

NASA  CR-2444,  Washington,  D.C.:  June  1974. 

24.  Laveson,  J.  I.  "NASA  Flight  Management  Research,"  Proceedings  of  the 
Human  Factors  Society  -  21st  Annual  Meeting  -  1977,  pp.  411-415. 


169 


25.  Lucaccini,  L.  Freedy,  A.  and  Hopf-Weichel ,  R.  Instructional  Sys¬ 
tem  Development  and  Evaluation  of  Situational  Emergency  Training 
(SET)  in  1979  Review  of  Air  Force  Sponsored  Basic  Research: 

Flight  and  Technical  Training.  Bolling  AFB,  D.C. :  AFOSR,  1979. 

26.  McCormick,  E.  J.  Human  Factors  Engineering.  New  York:  McGraw- 
Hill,  Inc.,  1964. 

27.  Mendenhall,  W.  A  Bibliography  on  Life  Testing  and  Related  Topics, 
Biometr ika,  Vol.  45,  1958,  pp.  521-543. 

28.  Meister,  D.  Human  Factors:  Theory  and  Practice.  New  York:  John 
Wiley  &  Sons,  Inc.,  1971. 

29.  _ ,  and  Rabideau,  G.  F.  Human  Factors  Evaluation  in  System 

Development.  New  York:  John  Wiley  &  Sons,  Inc.,  1965. 

30.  Miller,  R.  B.  Task  Taxonomy:  Science  or  Technology?  in  W.  T. 
Singleton,  R.  S.  Esterly,  and  D.  C.  Whitfield  (Eds.)  The  Human 
Operator  in  Complex  Systems.  London:  Taylor  and  Francis,  Ltd., 

1967. 

31.  _ .  A  Method  for  Determining  Task  Strategies  ,  AFHRL-TR-74-26 , 

Wright-Pattersson  Air  Force  Base,  Ohio:  Advanced  Systems  Division, 
May  1974. 

32.  Moray,  N.  Attention,  Control,  and  Sampling  Behavior  in  Monitoring 
Behavior  and  Supervisory  Control.  Sheridan,  T.  B.  and  Johannsen, 

G.,  Eds.  New  York:  Plenum  Press,  1976. 

33.  National  Transportation  Safety  Board,  Safety  Information  Bulletin 
SB  78-1,  Washington,  D.C.:  January  6,  1978. 

34.  _ .  Allegheny  Airlines  BAC  1-11,  Rochester,  New  York,  July  9, 

1978.  Report  No.  NTSB-AAR-79-2 .  Washington:  February  8,  1979. 

35 .  _ .  American  Airlines  DC-1010,  Chicago-O'Hare  International 

Airport,  May  25,  1979.  Report  No.  NTSB-AAR-79-17 ,  Washington: 
December  21,  1979. 

36.  _ .  Delta  Air  Lines  Douglas  DC-9-31,  Boston,  Massachusetts, 

July  31,  1973.  Report  No.  NTSB-AAR-74-3.  Washington:  March  7, 

1974. 

37.  _ .  Eastern  Air  Lines  L-1011,  Miami,  Florida,  December  29, 

1972.  Report  No.  NTSB-AAR-73-14 .  Washington:  June  14,  1973. 

38.  _ .  National  Airies  Boeing  727-235,  Escambia  Bay,  Pensacola, 

Florida,  May  8,  1978.  Report  No.  NTSB-AAR-78-13 ,  Washington: 
November  9,  1978. 


170 


39.  _ .  North  Central  Airlines  Convalr  580,  Kalamazoo,  Michigan, 

July  25,  1978.  Report  No.  NTSB-AAR-79-4 ,  Washington:  February  22, 
1979. 


40.  .  Northwest  Airlines  Boeing  727-251,  near  Thiells,  New 
York,  December  1,  1974.  Report  No.  NTSB-AAR-75-13 .  Washington: 
August  13,  1975. 

41.  _ .  Pacific  Southwest  Airlines  Boeing  727-214  and  Gibbs  Flite 

Center  Cessna  172,  San  Diego,  California,  September  25,  1978. 

Report  No.  NTSB-AAR-79-5 ,  Washington:  April  20,  1979. 

42.  _ .  Pan  American  World  Airways  Boeing  707-321C,  Boston, 

Massachusetts,  November  3,  1973.  Report  No.  NTSB-AAR-74-13T  Wash¬ 
ington:  December  2,  1974. 

43.  _ .  Southern  Airways  DC-9-31,  New  Hope,  Georgia,  April  4, 

1977.  Report  No.  NTSB-AAR-78-3 ,  Washington:  January  26,  1978. 

44.  _ .  Trans  World  Airlines  Boeing  727-231,  Berryville,  Virginia, 

December  1,  1974.  Report  No.  NTSB-AAR-75-16 ,  Washington:  November 
26,  1975. 

45.  _ .  United  Airlines  DC-9-54,  near  Kaysville,  Utah,  December 

18,  1977.  Report  No.  NTSB-AAR-78-8 ,  Washington:  July  27,  1978. 

46.  _ .  United  Airlines  DC-8-61,  Portland,  Oregon,  December  28, 

1978.  Report  No.  NTSB-AAR-79-7 ,  Washington:  June  7,  1979. 

47.  Neter,  J.  and  Wasserman,  W.  Applied  Linear  Statistical  Models. 
Homewood,  Illinois:  Richard  D.  Irwin,  Inc.,  1974. 

48.  Parsons,  H.  Mcl.  Man-Machine  System  Experiments.  Baltimore:  The 
Johns  Hopkins  Press,  1972. 

49.  Plan  to  Permit  Additional  Flightcrew  Training  in  Advanced  Flight 
Training  Simulators,  Federal  Register,  Vol.  44,  No.  220,  November 
13,  1979,  pp.  65550-65557. 

50.  Rasmussen,  J.  Outlines  of  a  Hybrid  Model  of  the  Process  Plant 
Operator  in  Monitoring  Behavior  and  Supervisory  Control. 

Sheridan,  T.  B.  and  Johannsen,  G. ,  Eds.  New  York:  Plenum  Press, 
1976. 

51.  Roscoe,  A.  H.  Stress  and  Workload  in  Pilots.  Aviation,  Space, 
and  Environmental  Medicine,  April,  1978. 

52.  Roscoe,  S.  Procedural  Doctrine  and  the  Precreation  of  Emergency 
Scenarios  in  Aircrew  Emergency  Dicision  Training:  A  Conference 
Report  November  28-30,  1978.  San  Francisco:  Perceptronics ,  Ind. , 


1979. 


171 


53.  Ruffell  Smith,  H.  P.  A  Simulator  Study  of  the  Interaction  of  Pilot 
Workload  with  Errors,  Vigilance,  and  Decisions.  NASA  TM-78482, 
Washington,  D.C. :  January  1979. 

54.  Shoulders,  G.  L.  Ants  vs.  Elephants.  Aerospace  Safety,  Vol.  35, 

No.  12,  pp.  2-3.  (December,  1979) 

55.  Singleton,  W.  T.  Theoretical  Approaches  to  Human  Error,  Ergonimics, 
Vol.  16,  No.  6,  1973,  pp.  727-737. 

56.  Smith,  R.  L.  Reducing  Pilot  Error  in  USAF  Aviation  Mishaps  in  Air 
Command  and  Staff  College  Research  Report  2375-78.  Maxwell  AFB, 
Alabama:  USAF,  May,  1978. 

57.  Sheridan,  T.  B.  Toward  a  General  Model  of  Supervisory  Control  in 
Monitoring  Behavior  and  Supervisory  Control.  Sheridan,  T.  B.  and 
Johannsen,  G. ,  Eds.  New  York:  Plenum  Press,  1976. 

58.  Teichner,  W.  H.  An  Information-Theoretica  Approach  to  Task  Classi¬ 
fication  Speech  at  77th  Annual  Meeting,  American  Psychological 
Association,  Washington,  D.C.,  September  1969. 

59.  Thrope,  J.  A.,  Martin,  E.  L.,  Edwards,  B.  J.,  and  Eddowes,  E.  E. 
Situational  Emergency  Training:  F-15  Emergency  Procedures  Training 
Program,  AFHRL-TR-76-47(l) ,  Williams  Air  Force  Base,  Arizona: 

Flying  Training  Division  June  1976. 

60.  Trollip,  S.  R.  The  Evaluation  of  a  Complex  Computer-Based  Flight 
Procedures  Trainer.  Human  Factors,  February,  1979,  pp.  47-54. 

61.  United  Press  Internationa,  "KLM  crash  at  Canary  Islands  took  most 
lives"  in  The  Columbus  Citizen-Journal,  November  29,  1979,  p.  4. 

62.  USAF  Management  Process,  Air  Force  Manual  25-1.  Washington: 
Department  of  the  Air  Force,  October  15,  1964. 

63.  Veldhuyzen,  W.  and  Stassen,  H.  G.  The  Internal  Model:  What  Does  It 
Mean  in  Human  Control  in  Monitoring  Behavior  and  Supervisory 
Control.  Sheridan,  T.  B.  and  Johannsen,  G.,  Eds.  New  York: 

Plenum  Press,  1976. 

64.  Vreuls,  D.  Aircrew  Performance  Measurement  Research  in  1979 
Review  of  Air  Force  Sponsored  Basic  Research:  Flight  and  Technical 
Training.  Bolling  AFB,  D. C. :  AFOSR,  1979. 

65.  Weber,  D.  C.  Accident  Rate  Potential:  An  Application  of  Multiple 
Regression  Analysis  of  a  Poisson  Process,  Journal  of  the  American 
Statistical  Association,  Vol.  66,  No.  334,  pp.  285-288.  (June, 

1971) 


66.  Weiner,  E.  E.  Controlled  Flight  into  Terrain  Accidents:  System- 
Induced  Errors,  Human  Factors,  Vol.  19,  No.  2,  pp.  171-181  (April, 
1977). 

67.  Wilkinson,  S.  Parallax:  Use  Your  Head,  Not  Your  Checklist. 

Flying,  Vol.  102,  No.  5,  p.  118  (May,  1978). 

68.  Williams,  A.  C.  and  Hopkins,  C.  0.  Aspects  of  Pilot  Decision 
Making,  WADC  TR-58-522,  Wright  Patterson  Air  Force  Base,  Ohio: 
December,  1958. 

69.  Williges,  B.  H.  ,  Roscoe,  S.  N. ,  and  Williges,  R.  C.  Synthetic 
Flight  Training  Revisited.  Human  Factors,  December,  1973,  pp. 
543-560. 


t 


